Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_whim_just_mpnn_from_auc/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_whim_just_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 4,
 'device': device(type='cuda', index=1),
 'dropout': 0.25,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': None,
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 500,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 500,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_whim_just_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': None,
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': None,
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': False,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.651643
Epoch 1
Validation auc = 0.629108
Epoch 2
Validation auc = 0.585915
Epoch 3
Validation auc = 0.565258
Epoch 4
Loss = 6.5449e-01, PNorm = 42.2865, GNorm = 1.0633, lr_0 = 6.5000e-04
Validation auc = 0.551643
Epoch 5
Validation auc = 0.544601
Epoch 6
Validation auc = 0.530986
Epoch 7
Validation auc = 0.555869
Epoch 8
Validation auc = 0.584977
Epoch 9
Loss = 6.3515e-01, PNorm = 42.3509, GNorm = 0.8689, lr_0 = 9.9743e-04
Validation auc = 0.643662
Epoch 10
Validation auc = 0.663850
Epoch 11
Validation auc = 0.661502
Epoch 12
Validation auc = 0.696714
Epoch 13
Validation auc = 0.670892
Epoch 14
Loss = 5.3929e-01, PNorm = 42.4542, GNorm = 1.0891, lr_0 = 9.8890e-04
Validation auc = 0.680282
Epoch 15
Validation auc = 0.702817
Epoch 16
Validation auc = 0.692958
Epoch 17
Validation auc = 0.738967
Epoch 18
Validation auc = 0.689671
Epoch 19
Loss = 5.7806e-01, PNorm = 42.5353, GNorm = 2.5958, lr_0 = 9.8045e-04
Validation auc = 0.699531
Epoch 20
Validation auc = 0.718310
Epoch 21
Validation auc = 0.705634
Epoch 22
Validation auc = 0.698592
Epoch 23
Validation auc = 0.727700
Epoch 24
Loss = 4.7060e-01, PNorm = 42.6162, GNorm = 0.6356, lr_0 = 9.7207e-04
Validation auc = 0.716901
Epoch 25
Validation auc = 0.723944
Epoch 26
Validation auc = 0.745540
Epoch 27
Validation auc = 0.727700
Epoch 28
Validation auc = 0.760563
Epoch 29
Loss = 5.2792e-01, PNorm = 42.6950, GNorm = 4.2137, lr_0 = 9.6376e-04
Validation auc = 0.753991
Epoch 30
Validation auc = 0.754460
Epoch 31
Validation auc = 0.765728
Epoch 32
Validation auc = 0.760563
Epoch 33
Validation auc = 0.740376
Epoch 34
Loss = 4.7043e-01, PNorm = 42.7731, GNorm = 2.6995, lr_0 = 9.5552e-04
Validation auc = 0.753052
Epoch 35
Validation auc = 0.733333
Epoch 36
Validation auc = 0.756338
Epoch 37
Validation auc = 0.763380
Epoch 38
Validation auc = 0.760563
Epoch 39
Loss = 4.5691e-01, PNorm = 42.8452, GNorm = 1.8189, lr_0 = 9.4735e-04
Validation auc = 0.777465
Epoch 40
Validation auc = 0.772770
Epoch 41
Validation auc = 0.765728
Epoch 42
Validation auc = 0.775587
Epoch 43
Validation auc = 0.780751
Epoch 44
Loss = 4.4525e-01, PNorm = 42.9232, GNorm = 0.9612, lr_0 = 9.3925e-04
Validation auc = 0.732394
Epoch 45
Validation auc = 0.758685
Epoch 46
Validation auc = 0.773239
Epoch 47
Validation auc = 0.762441
Epoch 48
Validation auc = 0.761033
Epoch 49
Loss = 3.8394e-01, PNorm = 43.0093, GNorm = 0.6886, lr_0 = 9.3122e-04
Validation auc = 0.768075
Epoch 50
Validation auc = 0.768075
Epoch 51
Validation auc = 0.759624
Epoch 52
Validation auc = 0.748826
Epoch 53
Validation auc = 0.749296
Epoch 54
Loss = 4.1641e-01, PNorm = 43.1023, GNorm = 2.6392, lr_0 = 9.2326e-04
Validation auc = 0.746948
Epoch 55
Validation auc = 0.735211
Epoch 56
Validation auc = 0.755399
Epoch 57
Validation auc = 0.724413
Epoch 58
Validation auc = 0.714554
Epoch 59
Loss = 3.6343e-01, PNorm = 43.1852, GNorm = 2.2279, lr_0 = 9.1537e-04
Validation auc = 0.721127
Epoch 60
Validation auc = 0.693897
Epoch 61
Validation auc = 0.703756
Epoch 62
Validation auc = 0.723474
Epoch 63
Validation auc = 0.734742
Epoch 64
Loss = 3.0462e-01, PNorm = 43.2867, GNorm = 1.0769, lr_0 = 9.0754e-04
Validation auc = 0.733803
Epoch 65
Validation auc = 0.736620
Epoch 66
Validation auc = 0.732864
Epoch 67
Validation auc = 0.722066
Epoch 68
Validation auc = 0.728169
Epoch 69
Loss = 2.6747e-01, PNorm = 43.3899, GNorm = 3.8051, lr_0 = 8.9978e-04
Validation auc = 0.702817
Epoch 70
Validation auc = 0.730986
Epoch 71
Validation auc = 0.692488
Epoch 72
Validation auc = 0.697653
Epoch 73
Validation auc = 0.704225
Epoch 74
Loss = 2.6594e-01, PNorm = 43.4834, GNorm = 1.2706, lr_0 = 8.9209e-04
Validation auc = 0.671831
Epoch 75
Validation auc = 0.676995
Epoch 76
Validation auc = 0.698592
Epoch 77
Validation auc = 0.692958
Epoch 78
Validation auc = 0.701878
Epoch 79
Loss = 2.4059e-01, PNorm = 43.5882, GNorm = 1.0288, lr_0 = 8.8447e-04
Validation auc = 0.729108
Epoch 80
Validation auc = 0.714554
Epoch 81
Validation auc = 0.695775
Epoch 82
Validation auc = 0.707512
Epoch 83
Validation auc = 0.701878
Epoch 84
Loss = 2.3772e-01, PNorm = 43.6960, GNorm = 3.7643, lr_0 = 8.7691e-04
Validation auc = 0.694836
Epoch 85
Validation auc = 0.700939
Epoch 86
Validation auc = 0.703756
Epoch 87
Validation auc = 0.694836
Epoch 88
Validation auc = 0.709859
Epoch 89
Loss = 2.1933e-01, PNorm = 43.7977, GNorm = 1.9005, lr_0 = 8.6941e-04
Validation auc = 0.699531
Epoch 90
Validation auc = 0.692019
Epoch 91
Validation auc = 0.689202
Epoch 92
Validation auc = 0.695775
Epoch 93
Validation auc = 0.686385
Epoch 94
Loss = 2.7497e-01, PNorm = 43.8972, GNorm = 3.4644, lr_0 = 8.6198e-04
Validation auc = 0.693427
Epoch 95
Validation auc = 0.686854
Epoch 96
Validation auc = 0.672770
Epoch 97
Validation auc = 0.676995
Epoch 98
Validation auc = 0.653052
Epoch 99
Loss = 2.0468e-01, PNorm = 43.9830, GNorm = 2.0210, lr_0 = 8.5461e-04
Validation auc = 0.669953
Epoch 100
Validation auc = 0.684507
Epoch 101
Validation auc = 0.676995
Epoch 102
Validation auc = 0.712676
Epoch 103
Validation auc = 0.708920
Epoch 104
Loss = 1.6594e-01, PNorm = 44.0727, GNorm = 2.6602, lr_0 = 8.4730e-04
Validation auc = 0.670892
Epoch 105
Validation auc = 0.676526
Epoch 106
Validation auc = 0.666667
Epoch 107
Validation auc = 0.657746
Epoch 108
Validation auc = 0.680751
Epoch 109
Loss = 2.0585e-01, PNorm = 44.1586, GNorm = 4.5610, lr_0 = 8.4006e-04
Validation auc = 0.701408
Epoch 110
Validation auc = 0.683568
Epoch 111
Validation auc = 0.670423
Epoch 112
Validation auc = 0.686385
Epoch 113
Validation auc = 0.676056
Epoch 114
Loss = 1.7234e-01, PNorm = 44.2486, GNorm = 1.2358, lr_0 = 8.3288e-04
Validation auc = 0.669953
Epoch 115
Validation auc = 0.679343
Epoch 116
Validation auc = 0.673709
Epoch 117
Validation auc = 0.655869
Epoch 118
Validation auc = 0.658685
Epoch 119
Loss = 1.8414e-01, PNorm = 44.3399, GNorm = 1.6229, lr_0 = 8.2576e-04
Validation auc = 0.673239
Epoch 120
Validation auc = 0.661502
Epoch 121
Validation auc = 0.652582
Epoch 122
Validation auc = 0.651643
Epoch 123
Validation auc = 0.659155
Epoch 124
Loss = 2.3290e-01, PNorm = 44.4169, GNorm = 4.7231, lr_0 = 8.1870e-04
Validation auc = 0.651643
Epoch 125
Validation auc = 0.646009
Epoch 126
Validation auc = 0.662911
Epoch 127
Validation auc = 0.660563
Epoch 128
Validation auc = 0.661972
Epoch 129
Loss = 1.2811e-01, PNorm = 44.4885, GNorm = 1.0673, lr_0 = 8.1170e-04
Validation auc = 0.675117
Epoch 130
Validation auc = 0.686385
Epoch 131
Validation auc = 0.677465
Epoch 132
Validation auc = 0.664789
Epoch 133
Validation auc = 0.689202
Epoch 134
Loss = 2.0564e-01, PNorm = 44.5624, GNorm = 7.5781, lr_0 = 8.0476e-04
Validation auc = 0.688732
Epoch 135
Validation auc = 0.681221
Epoch 136
Validation auc = 0.686385
Epoch 137
Validation auc = 0.699061
Epoch 138
Validation auc = 0.691080
Epoch 139
Loss = 1.7308e-01, PNorm = 44.6436, GNorm = 1.9477, lr_0 = 7.9788e-04
Validation auc = 0.675587
Epoch 140
Validation auc = 0.661502
Epoch 141
Validation auc = 0.656338
Epoch 142
Validation auc = 0.658685
Epoch 143
Validation auc = 0.661502
Epoch 144
Loss = 9.5276e-02, PNorm = 44.7196, GNorm = 0.7469, lr_0 = 7.9106e-04
Validation auc = 0.664789
Epoch 145
Validation auc = 0.665258
Epoch 146
Validation auc = 0.665258
Epoch 147
Validation auc = 0.669484
Epoch 148
Validation auc = 0.661502
Epoch 149
Loss = 3.9772e-02, PNorm = 44.7884, GNorm = 0.6617, lr_0 = 7.8430e-04
Validation auc = 0.656338
Epoch 150
Validation auc = 0.667136
Epoch 151
Validation auc = 0.674178
Epoch 152
Validation auc = 0.661033
Epoch 153
Validation auc = 0.664789
Epoch 154
Loss = 1.2862e-01, PNorm = 44.8520, GNorm = 3.1680, lr_0 = 7.7759e-04
Validation auc = 0.675587
Epoch 155
Validation auc = 0.647418
Epoch 156
Validation auc = 0.640376
Epoch 157
Validation auc = 0.667606
Epoch 158
Validation auc = 0.686385
Epoch 159
Loss = 1.3595e-01, PNorm = 44.9127, GNorm = 2.7362, lr_0 = 7.7095e-04
Validation auc = 0.693897
Epoch 160
Validation auc = 0.694366
Epoch 161
Validation auc = 0.696714
Epoch 162
Validation auc = 0.707042
Epoch 163
Validation auc = 0.713615
Epoch 164
Loss = 1.7421e-01, PNorm = 44.9777, GNorm = 2.7771, lr_0 = 7.6436e-04
Validation auc = 0.702817
Epoch 165
Validation auc = 0.680751
Epoch 166
Validation auc = 0.666667
Epoch 167
Validation auc = 0.664319
Epoch 168
Validation auc = 0.670892
Epoch 169
Loss = 7.1377e-02, PNorm = 45.0421, GNorm = 0.9084, lr_0 = 7.5782e-04
Validation auc = 0.672300
Epoch 170
Validation auc = 0.669484
Epoch 171
Validation auc = 0.664789
Epoch 172
Validation auc = 0.673709
Epoch 173
Validation auc = 0.678873
Epoch 174
Loss = 1.1684e-01, PNorm = 45.0983, GNorm = 3.0178, lr_0 = 7.5134e-04
Validation auc = 0.683099
Epoch 175
Validation auc = 0.677934
Epoch 176
Validation auc = 0.676995
Epoch 177
Validation auc = 0.678404
Epoch 178
Validation auc = 0.671362
Epoch 179
Loss = 9.0506e-02, PNorm = 45.1504, GNorm = 1.4431, lr_0 = 7.4492e-04
Validation auc = 0.667606
Epoch 180
Validation auc = 0.663850
Epoch 181
Validation auc = 0.666197
Epoch 182
Validation auc = 0.672770
Epoch 183
Validation auc = 0.679812
Epoch 184
Loss = 7.6574e-02, PNorm = 45.2085, GNorm = 1.3052, lr_0 = 7.3855e-04
Validation auc = 0.684507
Epoch 185
Validation auc = 0.688263
Epoch 186
Validation auc = 0.692958
Epoch 187
Validation auc = 0.695305
Epoch 188
Validation auc = 0.696714
Epoch 189
Loss = 4.1821e-02, PNorm = 45.2577, GNorm = 0.8479, lr_0 = 7.3224e-04
Validation auc = 0.696244
Epoch 190
Validation auc = 0.696714
Epoch 191
Validation auc = 0.702817
Epoch 192
Validation auc = 0.691549
Epoch 193
Validation auc = 0.682160
Epoch 194
Loss = 1.0697e-01, PNorm = 45.2994, GNorm = 1.0623, lr_0 = 7.2598e-04
Validation auc = 0.684038
Epoch 195
Validation auc = 0.682160
Epoch 196
Validation auc = 0.683099
Epoch 197
Validation auc = 0.680282
Epoch 198
Validation auc = 0.672770
Epoch 199
Loss = 9.0579e-02, PNorm = 45.3452, GNorm = 1.2681, lr_0 = 7.1977e-04
Validation auc = 0.665728
Epoch 200
Validation auc = 0.669953
Epoch 201
Validation auc = 0.678404
Epoch 202
Validation auc = 0.663850
Epoch 203
Validation auc = 0.654460
Epoch 204
Loss = 1.0254e-01, PNorm = 45.3947, GNorm = 3.8791, lr_0 = 7.1362e-04
Validation auc = 0.675117
Epoch 205
Validation auc = 0.692019
Epoch 206
Validation auc = 0.690610
Epoch 207
Validation auc = 0.692488
Epoch 208
Validation auc = 0.706573
Epoch 209
Loss = 9.0497e-02, PNorm = 45.4433, GNorm = 2.5347, lr_0 = 7.0752e-04
Validation auc = 0.717371
Epoch 210
Validation auc = 0.713146
Epoch 211
Validation auc = 0.710329
Epoch 212
Validation auc = 0.709390
Epoch 213
Validation auc = 0.708920
Epoch 214
Loss = 9.1071e-02, PNorm = 45.4995, GNorm = 2.0608, lr_0 = 7.0147e-04
Validation auc = 0.701408
Epoch 215
Validation auc = 0.690141
Epoch 216
Validation auc = 0.684507
Epoch 217
Validation auc = 0.688263
Epoch 218
Validation auc = 0.686385
Epoch 219
Loss = 2.6544e-02, PNorm = 45.5548, GNorm = 0.7264, lr_0 = 6.9548e-04
Validation auc = 0.684038
Epoch 220
Validation auc = 0.681221
Epoch 221
Validation auc = 0.676056
Epoch 222
Validation auc = 0.677934
Epoch 223
Validation auc = 0.690141
Epoch 224
Loss = 5.1274e-02, PNorm = 45.6024, GNorm = 1.3848, lr_0 = 6.8953e-04
Validation auc = 0.700000
Epoch 225
Validation auc = 0.681221
Epoch 226
Validation auc = 0.670423
Epoch 227
Validation auc = 0.668075
Epoch 228
Validation auc = 0.681221
Epoch 229
Loss = 3.7813e-02, PNorm = 45.6498, GNorm = 0.5235, lr_0 = 6.8364e-04
Validation auc = 0.686854
Epoch 230
Validation auc = 0.686854
Epoch 231
Validation auc = 0.679812
Epoch 232
Validation auc = 0.682629
Epoch 233
Validation auc = 0.685446
Epoch 234
Loss = 2.7528e-02, PNorm = 45.7005, GNorm = 0.8632, lr_0 = 6.7779e-04
Validation auc = 0.697653
Epoch 235
Validation auc = 0.701408
Epoch 236
Validation auc = 0.703756
Epoch 237
Validation auc = 0.703286
Epoch 238
Validation auc = 0.700469
Epoch 239
Loss = 8.1328e-02, PNorm = 45.7426, GNorm = 1.3595, lr_0 = 6.7200e-04
Validation auc = 0.699531
Epoch 240
Validation auc = 0.701878
Epoch 241
Validation auc = 0.699061
Epoch 242
Validation auc = 0.698122
Epoch 243
Validation auc = 0.693427
Epoch 244
Loss = 2.3864e-02, PNorm = 45.7758, GNorm = 0.6112, lr_0 = 6.6625e-04
Validation auc = 0.692958
Epoch 245
Validation auc = 0.694836
Epoch 246
Validation auc = 0.699061
Epoch 247
Validation auc = 0.697653
Epoch 248
Validation auc = 0.691080
Epoch 249
Loss = 7.3658e-02, PNorm = 45.8057, GNorm = 0.8569, lr_0 = 6.6056e-04
Validation auc = 0.684038
Epoch 250
Validation auc = 0.675117
Epoch 251
Validation auc = 0.675117
Epoch 252
Validation auc = 0.680751
Epoch 253
Validation auc = 0.686854
Epoch 254
Loss = 4.4184e-02, PNorm = 45.8416, GNorm = 0.7527, lr_0 = 6.5491e-04
Validation auc = 0.682629
Epoch 255
Validation auc = 0.676056
Epoch 256
Validation auc = 0.672770
Epoch 257
Validation auc = 0.669014
Epoch 258
Validation auc = 0.668075
Epoch 259
Loss = 5.0958e-02, PNorm = 45.8754, GNorm = 0.8119, lr_0 = 6.4931e-04
Validation auc = 0.670423
Epoch 260
Validation auc = 0.667606
Epoch 261
Validation auc = 0.665258
Epoch 262
Validation auc = 0.662441
Epoch 263
Validation auc = 0.664319
Epoch 264
Loss = 1.2197e-01, PNorm = 45.9011, GNorm = 1.3489, lr_0 = 6.4376e-04
Validation auc = 0.678873
Epoch 265
Validation auc = 0.686854
Epoch 266
Validation auc = 0.682160
Epoch 267
Validation auc = 0.675587
Epoch 268
Validation auc = 0.676526
Epoch 269
Loss = 1.3441e-02, PNorm = 45.9356, GNorm = 0.1356, lr_0 = 6.3826e-04
Validation auc = 0.669953
Epoch 270
Validation auc = 0.666197
Epoch 271
Validation auc = 0.666197
Epoch 272
Validation auc = 0.674648
Epoch 273
Validation auc = 0.695775
Epoch 274
Loss = 4.0841e-02, PNorm = 45.9667, GNorm = 1.8747, lr_0 = 6.3280e-04
Validation auc = 0.703756
Epoch 275
Validation auc = 0.706573
Epoch 276
Validation auc = 0.703756
Epoch 277
Validation auc = 0.710798
Epoch 278
Validation auc = 0.711737
Epoch 279
Loss = 3.6606e-02, PNorm = 46.0052, GNorm = 0.4007, lr_0 = 6.2739e-04
Validation auc = 0.706103
Epoch 280
Validation auc = 0.693897
Epoch 281
Validation auc = 0.686854
Epoch 282
Validation auc = 0.680282
Epoch 283
Validation auc = 0.685446
Epoch 284
Loss = 1.6066e-02, PNorm = 46.0421, GNorm = 0.5615, lr_0 = 6.2203e-04
Validation auc = 0.684977
Epoch 285
Validation auc = 0.674648
Epoch 286
Validation auc = 0.662911
Epoch 287
Validation auc = 0.664319
Epoch 288
Validation auc = 0.667136
Epoch 289
Loss = 4.2907e-02, PNorm = 46.0771, GNorm = 2.8193, lr_0 = 6.1671e-04
Validation auc = 0.664789
Epoch 290
Validation auc = 0.661972
Epoch 291
Validation auc = 0.665258
Epoch 292
Validation auc = 0.687324
Epoch 293
Validation auc = 0.700469
Epoch 294
Loss = 6.3863e-02, PNorm = 46.1091, GNorm = 3.2450, lr_0 = 6.1144e-04
Validation auc = 0.703286
Epoch 295
Validation auc = 0.703756
Epoch 296
Validation auc = 0.705164
Epoch 297
Validation auc = 0.703756
Epoch 298
Validation auc = 0.707042
Epoch 299
Loss = 2.4395e-02, PNorm = 46.1482, GNorm = 0.7253, lr_0 = 6.0621e-04
Validation auc = 0.706573
Model 0 best validation auc = 0.780751 on epoch 43
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.889223
Ensemble test auc = 0.889223
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.602347
Epoch 1
Validation auc = 0.608920
Epoch 2
Validation auc = 0.620188
Epoch 3
Validation auc = 0.599531
Epoch 4
Loss = 6.7477e-01, PNorm = 42.2866, GNorm = 0.8864, lr_0 = 6.5000e-04
Validation auc = 0.574648
Epoch 5
Validation auc = 0.578873
Epoch 6
Validation auc = 0.627700
Epoch 7
Validation auc = 0.628638
Epoch 8
Validation auc = 0.652113
Epoch 9
Loss = 6.6855e-01, PNorm = 42.3567, GNorm = 1.8320, lr_0 = 9.9743e-04
Validation auc = 0.653991
Epoch 10
Validation auc = 0.659624
Epoch 11
Validation auc = 0.681690
Epoch 12
Validation auc = 0.694366
Epoch 13
Validation auc = 0.715023
Epoch 14
Loss = 6.0513e-01, PNorm = 42.4564, GNorm = 3.9264, lr_0 = 9.8890e-04
Validation auc = 0.708920
Epoch 15
Validation auc = 0.690141
Epoch 16
Validation auc = 0.681690
Epoch 17
Validation auc = 0.692488
Epoch 18
Validation auc = 0.694366
Epoch 19
Loss = 5.2437e-01, PNorm = 42.5372, GNorm = 0.5848, lr_0 = 9.8045e-04
Validation auc = 0.686854
Epoch 20
Validation auc = 0.715493
Epoch 21
Validation auc = 0.713615
Epoch 22
Validation auc = 0.715493
Epoch 23
Validation auc = 0.751643
Epoch 24
Loss = 5.2177e-01, PNorm = 42.6251, GNorm = 1.2007, lr_0 = 9.7207e-04
Validation auc = 0.739437
Epoch 25
Validation auc = 0.741315
Epoch 26
Validation auc = 0.760563
Epoch 27
Validation auc = 0.732394
Epoch 28
Validation auc = 0.756338
Epoch 29
Loss = 4.9235e-01, PNorm = 42.6981, GNorm = 1.9968, lr_0 = 9.6376e-04
Validation auc = 0.729108
Epoch 30
Validation auc = 0.717840
Epoch 31
Validation auc = 0.752113
Epoch 32
Validation auc = 0.776526
Epoch 33
Validation auc = 0.743662
Epoch 34
Loss = 4.8398e-01, PNorm = 42.7740, GNorm = 1.9879, lr_0 = 9.5552e-04
Validation auc = 0.755869
Epoch 35
Validation auc = 0.772770
Epoch 36
Validation auc = 0.768075
Epoch 37
Validation auc = 0.742254
Epoch 38
Validation auc = 0.761972
Epoch 39
Loss = 4.5805e-01, PNorm = 42.8540, GNorm = 1.9603, lr_0 = 9.4735e-04
Validation auc = 0.778873
Epoch 40
Validation auc = 0.761502
Epoch 41
Validation auc = 0.754460
Epoch 42
Validation auc = 0.756808
Epoch 43
Validation auc = 0.761502
Epoch 44
Loss = 3.9439e-01, PNorm = 42.9311, GNorm = 0.7790, lr_0 = 9.3925e-04
Validation auc = 0.761972
Epoch 45
Validation auc = 0.751174
Epoch 46
Validation auc = 0.763380
Epoch 47
Validation auc = 0.781690
Epoch 48
Validation auc = 0.782629
Epoch 49
Loss = 3.7973e-01, PNorm = 43.0149, GNorm = 1.4098, lr_0 = 9.3122e-04
Validation auc = 0.770892
Epoch 50
Validation auc = 0.775117
Epoch 51
Validation auc = 0.767136
Epoch 52
Validation auc = 0.756808
Epoch 53
Validation auc = 0.776995
Epoch 54
Loss = 4.3815e-01, PNorm = 43.0956, GNorm = 3.3333, lr_0 = 9.2326e-04
Validation auc = 0.753991
Epoch 55
Validation auc = 0.753521
Epoch 56
Validation auc = 0.755399
Epoch 57
Validation auc = 0.750235
Epoch 58
Validation auc = 0.745070
Epoch 59
Loss = 3.1422e-01, PNorm = 43.1849, GNorm = 2.0911, lr_0 = 9.1537e-04
Validation auc = 0.743662
Epoch 60
Validation auc = 0.730516
Epoch 61
Validation auc = 0.742254
Epoch 62
Validation auc = 0.732864
Epoch 63
Validation auc = 0.742254
Epoch 64
Loss = 3.0829e-01, PNorm = 43.2840, GNorm = 1.4387, lr_0 = 9.0754e-04
Validation auc = 0.735211
Epoch 65
Validation auc = 0.722535
Epoch 66
Validation auc = 0.727700
Epoch 67
Validation auc = 0.710329
Epoch 68
Validation auc = 0.707042
Epoch 69
Loss = 2.7504e-01, PNorm = 43.3856, GNorm = 2.0090, lr_0 = 8.9978e-04
Validation auc = 0.711737
Epoch 70
Validation auc = 0.712207
Epoch 71
Validation auc = 0.706103
Epoch 72
Validation auc = 0.715023
Epoch 73
Validation auc = 0.722066
Epoch 74
Loss = 3.2684e-01, PNorm = 43.4854, GNorm = 7.6475, lr_0 = 8.9209e-04
Validation auc = 0.718310
Epoch 75
Validation auc = 0.723944
Epoch 76
Validation auc = 0.723005
Epoch 77
Validation auc = 0.695305
Epoch 78
Validation auc = 0.714554
Epoch 79
Loss = 2.2640e-01, PNorm = 43.5816, GNorm = 2.3133, lr_0 = 8.8447e-04
Validation auc = 0.708920
Epoch 80
Validation auc = 0.704695
Epoch 81
Validation auc = 0.711268
Epoch 82
Validation auc = 0.706573
Epoch 83
Validation auc = 0.689671
Epoch 84
Loss = 1.9042e-01, PNorm = 43.6880, GNorm = 1.7503, lr_0 = 8.7691e-04
Validation auc = 0.700939
Epoch 85
Validation auc = 0.677465
Epoch 86
Validation auc = 0.704695
Epoch 87
Validation auc = 0.704225
Epoch 88
Validation auc = 0.695775
Epoch 89
Loss = 2.4121e-01, PNorm = 43.7805, GNorm = 0.9503, lr_0 = 8.6941e-04
Validation auc = 0.717371
Epoch 90
Validation auc = 0.711268
Epoch 91
Validation auc = 0.715023
Epoch 92
Validation auc = 0.718779
Epoch 93
Validation auc = 0.700000
Epoch 94
Loss = 2.5872e-01, PNorm = 43.8749, GNorm = 2.5546, lr_0 = 8.6198e-04
Validation auc = 0.706573
Epoch 95
Validation auc = 0.708920
Epoch 96
Validation auc = 0.687793
Epoch 97
Validation auc = 0.702817
Epoch 98
Validation auc = 0.708920
Epoch 99
Loss = 1.3726e-01, PNorm = 43.9811, GNorm = 0.9987, lr_0 = 8.5461e-04
Validation auc = 0.696714
Epoch 100
Validation auc = 0.707512
Epoch 101
Validation auc = 0.710798
Epoch 102
Validation auc = 0.693897
Epoch 103
Validation auc = 0.698122
Epoch 104
Loss = 2.1315e-01, PNorm = 44.0729, GNorm = 2.9310, lr_0 = 8.4730e-04
Validation auc = 0.705634
Epoch 105
Validation auc = 0.684038
Epoch 106
Validation auc = 0.676526
Epoch 107
Validation auc = 0.703756
Epoch 108
Validation auc = 0.686854
Epoch 109
Loss = 1.6298e-01, PNorm = 44.1579, GNorm = 3.8106, lr_0 = 8.4006e-04
Validation auc = 0.684977
Epoch 110
Validation auc = 0.712676
Epoch 111
Validation auc = 0.702817
Epoch 112
Validation auc = 0.683568
Epoch 113
Validation auc = 0.695775
Epoch 114
Loss = 1.7557e-01, PNorm = 44.2346, GNorm = 5.5837, lr_0 = 8.3288e-04
Validation auc = 0.702817
Epoch 115
Validation auc = 0.687793
Epoch 116
Validation auc = 0.684507
Epoch 117
Validation auc = 0.696244
Epoch 118
Validation auc = 0.700000
Epoch 119
Loss = 1.6513e-01, PNorm = 44.3188, GNorm = 1.8811, lr_0 = 8.2576e-04
Validation auc = 0.699531
Epoch 120
Validation auc = 0.707042
Epoch 121
Validation auc = 0.704695
Epoch 122
Validation auc = 0.694366
Epoch 123
Validation auc = 0.694836
Epoch 124
Loss = 1.1908e-01, PNorm = 44.4062, GNorm = 1.6520, lr_0 = 8.1870e-04
Validation auc = 0.694836
Epoch 125
Validation auc = 0.692019
Epoch 126
Validation auc = 0.696714
Epoch 127
Validation auc = 0.721596
Epoch 128
Validation auc = 0.703756
Epoch 129
Loss = 1.8268e-01, PNorm = 44.4958, GNorm = 4.7534, lr_0 = 8.1170e-04
Validation auc = 0.689202
Epoch 130
Validation auc = 0.725352
Epoch 131
Validation auc = 0.709859
Epoch 132
Validation auc = 0.701878
Epoch 133
Validation auc = 0.711268
Epoch 134
Loss = 1.4392e-01, PNorm = 44.5821, GNorm = 3.8842, lr_0 = 8.0476e-04
Validation auc = 0.728638
Epoch 135
Validation auc = 0.711737
Epoch 136
Validation auc = 0.707981
Epoch 137
Validation auc = 0.709859
Epoch 138
Validation auc = 0.699531
Epoch 139
Loss = 4.8575e-02, PNorm = 44.6666, GNorm = 0.9546, lr_0 = 7.9788e-04
Validation auc = 0.696244
Epoch 140
Validation auc = 0.701408
Epoch 141
Validation auc = 0.702817
Epoch 142
Validation auc = 0.700939
Epoch 143
Validation auc = 0.699061
Epoch 144
Loss = 2.0160e-01, PNorm = 44.7390, GNorm = 3.8320, lr_0 = 7.9106e-04
Validation auc = 0.706103
Epoch 145
Validation auc = 0.710798
Epoch 146
Validation auc = 0.697653
Epoch 147
Validation auc = 0.694366
Epoch 148
Validation auc = 0.704225
Epoch 149
Loss = 1.3485e-01, PNorm = 44.7940, GNorm = 3.8407, lr_0 = 7.8430e-04
Validation auc = 0.708451
Epoch 150
Validation auc = 0.702347
Epoch 151
Validation auc = 0.705634
Epoch 152
Validation auc = 0.726291
Epoch 153
Validation auc = 0.712207
Epoch 154
Loss = 1.2044e-01, PNorm = 44.8550, GNorm = 3.6278, lr_0 = 7.7759e-04
Validation auc = 0.694366
Epoch 155
Validation auc = 0.701878
Epoch 156
Validation auc = 0.721127
Epoch 157
Validation auc = 0.696714
Epoch 158
Validation auc = 0.660094
Epoch 159
Loss = 2.1584e-01, PNorm = 44.9190, GNorm = 3.7218, lr_0 = 7.7095e-04
Validation auc = 0.659624
Epoch 160
Validation auc = 0.677934
Epoch 161
Validation auc = 0.682629
Epoch 162
Validation auc = 0.680282
Epoch 163
Validation auc = 0.675587
Epoch 164
Loss = 1.7288e-01, PNorm = 44.9877, GNorm = 2.9784, lr_0 = 7.6436e-04
Validation auc = 0.680282
Epoch 165
Validation auc = 0.699531
Epoch 166
Validation auc = 0.704695
Epoch 167
Validation auc = 0.700000
Epoch 168
Validation auc = 0.694836
Epoch 169
Loss = 8.8261e-02, PNorm = 45.0520, GNorm = 0.7997, lr_0 = 7.5782e-04
Validation auc = 0.700469
Epoch 170
Validation auc = 0.716432
Epoch 171
Validation auc = 0.719249
Epoch 172
Validation auc = 0.722066
Epoch 173
Validation auc = 0.710798
Epoch 174
Loss = 1.1862e-01, PNorm = 45.1064, GNorm = 2.1148, lr_0 = 7.5134e-04
Validation auc = 0.708451
Epoch 175
Validation auc = 0.712207
Epoch 176
Validation auc = 0.710329
Epoch 177
Validation auc = 0.703756
Epoch 178
Validation auc = 0.699061
Epoch 179
Loss = 9.9807e-02, PNorm = 45.1596, GNorm = 2.1301, lr_0 = 7.4492e-04
Validation auc = 0.692958
Epoch 180
Validation auc = 0.701408
Epoch 181
Validation auc = 0.700000
Epoch 182
Validation auc = 0.693897
Epoch 183
Validation auc = 0.696244
Epoch 184
Loss = 9.6671e-02, PNorm = 45.2158, GNorm = 5.6005, lr_0 = 7.3855e-04
Validation auc = 0.687793
Epoch 185
Validation auc = 0.675117
Epoch 186
Validation auc = 0.683568
Epoch 187
Validation auc = 0.697653
Epoch 188
Validation auc = 0.703286
Epoch 189
Loss = 1.2781e-01, PNorm = 45.2750, GNorm = 2.4635, lr_0 = 7.3224e-04
Validation auc = 0.703756
Epoch 190
Validation auc = 0.700939
Epoch 191
Validation auc = 0.703756
Epoch 192
Validation auc = 0.703756
Epoch 193
Validation auc = 0.697183
Epoch 194
Loss = 6.9860e-02, PNorm = 45.3278, GNorm = 2.2666, lr_0 = 7.2598e-04
Validation auc = 0.693427
Epoch 195
Validation auc = 0.696244
Epoch 196
Validation auc = 0.693427
Epoch 197
Validation auc = 0.680282
Epoch 198
Validation auc = 0.671831
Epoch 199
Loss = 5.0377e-02, PNorm = 45.3722, GNorm = 0.6543, lr_0 = 7.1977e-04
Validation auc = 0.687324
Epoch 200
Validation auc = 0.687324
Epoch 201
Validation auc = 0.689671
Epoch 202
Validation auc = 0.691549
Epoch 203
Validation auc = 0.690610
Epoch 204
Loss = 2.4298e-02, PNorm = 45.4112, GNorm = 0.6061, lr_0 = 7.1362e-04
Validation auc = 0.690610
Epoch 205
Validation auc = 0.686385
Epoch 206
Validation auc = 0.690141
Epoch 207
Validation auc = 0.690141
Epoch 208
Validation auc = 0.683568
Epoch 209
Loss = 4.3418e-02, PNorm = 45.4518, GNorm = 1.6236, lr_0 = 7.0752e-04
Validation auc = 0.684038
Epoch 210
Validation auc = 0.691080
Epoch 211
Validation auc = 0.701408
Epoch 212
Validation auc = 0.705164
Epoch 213
Validation auc = 0.694366
Epoch 214
Loss = 1.4258e-02, PNorm = 45.4991, GNorm = 0.7951, lr_0 = 7.0147e-04
Validation auc = 0.686385
Epoch 215
Validation auc = 0.698122
Epoch 216
Validation auc = 0.707512
Epoch 217
Validation auc = 0.702347
Epoch 218
Validation auc = 0.696244
Epoch 219
Loss = 6.6082e-02, PNorm = 45.5389, GNorm = 0.6897, lr_0 = 6.9548e-04
Validation auc = 0.694836
Epoch 220
Validation auc = 0.692019
Epoch 221
Validation auc = 0.689671
Epoch 222
Validation auc = 0.687324
Epoch 223
Validation auc = 0.688263
Epoch 224
Loss = 4.6588e-02, PNorm = 45.5770, GNorm = 1.8175, lr_0 = 6.8953e-04
Validation auc = 0.685915
Epoch 225
Validation auc = 0.677465
Epoch 226
Validation auc = 0.675587
Epoch 227
Validation auc = 0.680282
Epoch 228
Validation auc = 0.687793
Epoch 229
Loss = 3.7562e-02, PNorm = 45.6180, GNorm = 0.9305, lr_0 = 6.8364e-04
Validation auc = 0.689671
Epoch 230
Validation auc = 0.679343
Epoch 231
Validation auc = 0.682629
Epoch 232
Validation auc = 0.694366
Epoch 233
Validation auc = 0.703756
Epoch 234
Loss = 3.0456e-02, PNorm = 45.6623, GNorm = 1.5894, lr_0 = 6.7779e-04
Validation auc = 0.703286
Epoch 235
Validation auc = 0.691080
Epoch 236
Validation auc = 0.685446
Epoch 237
Validation auc = 0.680282
Epoch 238
Validation auc = 0.669014
Epoch 239
Loss = 4.4741e-02, PNorm = 45.7024, GNorm = 1.0928, lr_0 = 6.7200e-04
Validation auc = 0.671362
Epoch 240
Validation auc = 0.683568
Epoch 241
Validation auc = 0.692488
Epoch 242
Validation auc = 0.696714
Epoch 243
Validation auc = 0.699061
Epoch 244
Loss = 7.3964e-02, PNorm = 45.7465, GNorm = 2.7930, lr_0 = 6.6625e-04
Validation auc = 0.706573
Epoch 245
Validation auc = 0.691080
Epoch 246
Validation auc = 0.682629
Epoch 247
Validation auc = 0.684038
Epoch 248
Validation auc = 0.690610
Epoch 249
Loss = 1.1289e-01, PNorm = 45.7996, GNorm = 9.0358, lr_0 = 6.6056e-04
Validation auc = 0.678873
Epoch 250
Validation auc = 0.660563
Epoch 251
Validation auc = 0.663850
Epoch 252
Validation auc = 0.704695
Epoch 253
Validation auc = 0.707042
Epoch 254
Loss = 4.5401e-02, PNorm = 45.8505, GNorm = 0.5876, lr_0 = 6.5491e-04
Validation auc = 0.692019
Epoch 255
Validation auc = 0.691080
Epoch 256
Validation auc = 0.699531
Epoch 257
Validation auc = 0.701878
Epoch 258
Validation auc = 0.707042
Epoch 259
Loss = 8.3209e-02, PNorm = 45.9124, GNorm = 1.8282, lr_0 = 6.4931e-04
Validation auc = 0.704225
Epoch 260
Validation auc = 0.695305
Epoch 261
Validation auc = 0.695775
Epoch 262
Validation auc = 0.704225
Epoch 263
Validation auc = 0.715962
Epoch 264
Loss = 5.2286e-02, PNorm = 45.9698, GNorm = 1.4192, lr_0 = 6.4376e-04
Validation auc = 0.716432
Epoch 265
Validation auc = 0.706103
Epoch 266
Validation auc = 0.700939
Epoch 267
Validation auc = 0.697653
Epoch 268
Validation auc = 0.704695
Epoch 269
Loss = 1.0695e-02, PNorm = 46.0155, GNorm = 0.7572, lr_0 = 6.3826e-04
Validation auc = 0.701878
Epoch 270
Validation auc = 0.700000
Epoch 271
Validation auc = 0.699061
Epoch 272
Validation auc = 0.698592
Epoch 273
Validation auc = 0.697653
Epoch 274
Loss = 3.5401e-02, PNorm = 46.0540, GNorm = 2.1075, lr_0 = 6.3280e-04
Validation auc = 0.705634
Epoch 275
Validation auc = 0.712207
Epoch 276
Validation auc = 0.714085
Epoch 277
Validation auc = 0.710798
Epoch 278
Validation auc = 0.710329
Epoch 279
Loss = 3.2991e-02, PNorm = 46.0906, GNorm = 1.2347, lr_0 = 6.2739e-04
Validation auc = 0.712207
Epoch 280
Validation auc = 0.711268
Epoch 281
Validation auc = 0.708920
Epoch 282
Validation auc = 0.704225
Epoch 283
Validation auc = 0.707981
Epoch 284
Loss = 1.5636e-02, PNorm = 46.1191, GNorm = 1.0996, lr_0 = 6.2203e-04
Validation auc = 0.707042
Epoch 285
Validation auc = 0.709859
Epoch 286
Validation auc = 0.708920
Epoch 287
Validation auc = 0.704695
Epoch 288
Validation auc = 0.702817
Epoch 289
Loss = 8.5591e-03, PNorm = 46.1457, GNorm = 0.8545, lr_0 = 6.1671e-04
Validation auc = 0.700000
Epoch 290
Validation auc = 0.694836
Epoch 291
Validation auc = 0.685915
Epoch 292
Validation auc = 0.691549
Epoch 293
Validation auc = 0.691549
Epoch 294
Loss = 5.0526e-02, PNorm = 46.1791, GNorm = 1.2562, lr_0 = 6.1144e-04
Validation auc = 0.676526
Epoch 295
Validation auc = 0.670423
Epoch 296
Validation auc = 0.669484
Epoch 297
Validation auc = 0.687324
Epoch 298
Validation auc = 0.703756
Epoch 299
Loss = 4.1281e-02, PNorm = 46.2101, GNorm = 1.9139, lr_0 = 6.0621e-04
Validation auc = 0.707512
Model 0 best validation auc = 0.782629 on epoch 48
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.892234
Ensemble test auc = 0.892234
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.563850
Epoch 1
Validation auc = 0.543192
Epoch 2
Validation auc = 0.544131
Epoch 3
Validation auc = 0.521596
Epoch 4
Loss = 6.8870e-01, PNorm = 42.2865, GNorm = 1.1602, lr_0 = 6.5000e-04
Validation auc = 0.528638
Epoch 5
Validation auc = 0.564319
Epoch 6
Validation auc = 0.603286
Epoch 7
Validation auc = 0.621127
Epoch 8
Validation auc = 0.622535
Epoch 9
Loss = 6.3387e-01, PNorm = 42.3574, GNorm = 0.9442, lr_0 = 9.9743e-04
Validation auc = 0.627700
Epoch 10
Validation auc = 0.620657
Epoch 11
Validation auc = 0.655399
Epoch 12
Validation auc = 0.669953
Epoch 13
Validation auc = 0.687324
Epoch 14
Loss = 5.5579e-01, PNorm = 42.4555, GNorm = 1.4493, lr_0 = 9.8890e-04
Validation auc = 0.683099
Epoch 15
Validation auc = 0.698592
Epoch 16
Validation auc = 0.668545
Epoch 17
Validation auc = 0.723005
Epoch 18
Validation auc = 0.688732
Epoch 19
Loss = 6.0032e-01, PNorm = 42.5399, GNorm = 0.9648, lr_0 = 9.8045e-04
Validation auc = 0.729577
Epoch 20
Validation auc = 0.728169
Epoch 21
Validation auc = 0.704225
Epoch 22
Validation auc = 0.744601
Epoch 23
Validation auc = 0.743662
Epoch 24
Loss = 4.9067e-01, PNorm = 42.6198, GNorm = 0.9225, lr_0 = 9.7207e-04
Validation auc = 0.737559
Epoch 25
Validation auc = 0.746479
Epoch 26
Validation auc = 0.757277
Epoch 27
Validation auc = 0.734272
Epoch 28
Validation auc = 0.742723
Epoch 29
Loss = 5.5905e-01, PNorm = 42.6971, GNorm = 2.2286, lr_0 = 9.6376e-04
Validation auc = 0.763380
Epoch 30
Validation auc = 0.743192
Epoch 31
Validation auc = 0.753521
Epoch 32
Validation auc = 0.769484
Epoch 33
Validation auc = 0.765728
Epoch 34
Loss = 5.0965e-01, PNorm = 42.7713, GNorm = 1.6603, lr_0 = 9.5552e-04
Validation auc = 0.757277
Epoch 35
Validation auc = 0.769953
Epoch 36
Validation auc = 0.768075
Epoch 37
Validation auc = 0.755399
Epoch 38
Validation auc = 0.746009
Epoch 39
Loss = 4.6459e-01, PNorm = 42.8612, GNorm = 2.9960, lr_0 = 9.4735e-04
Validation auc = 0.774648
Epoch 40
Validation auc = 0.769953
Epoch 41
Validation auc = 0.748357
Epoch 42
Validation auc = 0.774178
Epoch 43
Validation auc = 0.787324
Epoch 44
Loss = 5.3866e-01, PNorm = 42.9385, GNorm = 1.1887, lr_0 = 9.3925e-04
Validation auc = 0.776056
Epoch 45
Validation auc = 0.765258
Epoch 46
Validation auc = 0.771362
Epoch 47
Validation auc = 0.781221
Epoch 48
Validation auc = 0.782160
Epoch 49
Loss = 4.3445e-01, PNorm = 43.0384, GNorm = 1.4763, lr_0 = 9.3122e-04
Validation auc = 0.767136
Epoch 50
Validation auc = 0.774178
Epoch 51
Validation auc = 0.777934
Epoch 52
Validation auc = 0.779812
Epoch 53
Validation auc = 0.771362
Epoch 54
Loss = 3.7229e-01, PNorm = 43.1259, GNorm = 1.0001, lr_0 = 9.2326e-04
Validation auc = 0.775117
Epoch 55
Validation auc = 0.784038
Epoch 56
Validation auc = 0.766197
Epoch 57
Validation auc = 0.753052
Epoch 58
Validation auc = 0.781221
Epoch 59
Loss = 4.8158e-01, PNorm = 43.2014, GNorm = 2.3354, lr_0 = 9.1537e-04
Validation auc = 0.767606
Epoch 60
Validation auc = 0.761502
Epoch 61
Validation auc = 0.773709
Epoch 62
Validation auc = 0.782629
Epoch 63
Validation auc = 0.771362
Epoch 64
Loss = 3.4741e-01, PNorm = 43.2751, GNorm = 1.4772, lr_0 = 9.0754e-04
Validation auc = 0.754460
Epoch 65
Validation auc = 0.763380
Epoch 66
Validation auc = 0.778404
Epoch 67
Validation auc = 0.773239
Epoch 68
Validation auc = 0.761502
Epoch 69
Loss = 3.3096e-01, PNorm = 43.3720, GNorm = 2.3077, lr_0 = 8.9978e-04
Validation auc = 0.760563
Epoch 70
Validation auc = 0.767136
Epoch 71
Validation auc = 0.761502
Epoch 72
Validation auc = 0.750235
Epoch 73
Validation auc = 0.767606
Epoch 74
Loss = 2.6355e-01, PNorm = 43.4609, GNorm = 1.7089, lr_0 = 8.9209e-04
Validation auc = 0.769014
Epoch 75
Validation auc = 0.766667
Epoch 76
Validation auc = 0.771362
Epoch 77
Validation auc = 0.761972
Epoch 78
Validation auc = 0.758685
Epoch 79
Loss = 3.2848e-01, PNorm = 43.5582, GNorm = 1.0862, lr_0 = 8.8447e-04
Validation auc = 0.760094
Epoch 80
Validation auc = 0.735211
Epoch 81
Validation auc = 0.730516
Epoch 82
Validation auc = 0.737089
Epoch 83
Validation auc = 0.728169
Epoch 84
Loss = 3.9847e-01, PNorm = 43.6496, GNorm = 2.7073, lr_0 = 8.7691e-04
Validation auc = 0.722066
Epoch 85
Validation auc = 0.730516
Epoch 86
Validation auc = 0.733803
Epoch 87
Validation auc = 0.723474
Epoch 88
Validation auc = 0.728638
Epoch 89
Loss = 2.7205e-01, PNorm = 43.7561, GNorm = 0.6712, lr_0 = 8.6941e-04
Validation auc = 0.739906
Epoch 90
Validation auc = 0.734742
Epoch 91
Validation auc = 0.727230
Epoch 92
Validation auc = 0.740845
Epoch 93
Validation auc = 0.741784
Epoch 94
Loss = 2.3723e-01, PNorm = 43.8740, GNorm = 0.8089, lr_0 = 8.6198e-04
Validation auc = 0.724883
Epoch 95
Validation auc = 0.715962
Epoch 96
Validation auc = 0.707512
Epoch 97
Validation auc = 0.697183
Epoch 98
Validation auc = 0.700469
Epoch 99
Loss = 2.5191e-01, PNorm = 43.9752, GNorm = 2.4511, lr_0 = 8.5461e-04
Validation auc = 0.720188
Epoch 100
Validation auc = 0.721596
Epoch 101
Validation auc = 0.713615
Epoch 102
Validation auc = 0.707042
Epoch 103
Validation auc = 0.719718
Epoch 104
Loss = 2.8956e-01, PNorm = 44.0867, GNorm = 1.9377, lr_0 = 8.4730e-04
Validation auc = 0.715493
Epoch 105
Validation auc = 0.722066
Epoch 106
Validation auc = 0.712207
Epoch 107
Validation auc = 0.694366
Epoch 108
Validation auc = 0.682629
Epoch 109
Loss = 2.0309e-01, PNorm = 44.1934, GNorm = 1.3701, lr_0 = 8.4006e-04
Validation auc = 0.683568
Epoch 110
Validation auc = 0.695305
Epoch 111
Validation auc = 0.707512
Epoch 112
Validation auc = 0.705164
Epoch 113
Validation auc = 0.697183
Epoch 114
Loss = 1.6006e-01, PNorm = 44.3066, GNorm = 4.8059, lr_0 = 8.3288e-04
Validation auc = 0.699061
Epoch 115
Validation auc = 0.694836
Epoch 116
Validation auc = 0.701878
Epoch 117
Validation auc = 0.701408
Epoch 118
Validation auc = 0.683568
Epoch 119
Loss = 1.3860e-01, PNorm = 44.4067, GNorm = 1.6444, lr_0 = 8.2576e-04
Validation auc = 0.672770
Epoch 120
Validation auc = 0.679343
Epoch 121
Validation auc = 0.684038
Epoch 122
Validation auc = 0.693427
Epoch 123
Validation auc = 0.697653
Epoch 124
Loss = 2.0398e-01, PNorm = 44.5131, GNorm = 5.4353, lr_0 = 8.1870e-04
Validation auc = 0.707512
Epoch 125
Validation auc = 0.702817
Epoch 126
Validation auc = 0.713146
Epoch 127
Validation auc = 0.716432
Epoch 128
Validation auc = 0.702347
Epoch 129
Loss = 1.3341e-01, PNorm = 44.6110, GNorm = 1.2120, lr_0 = 8.1170e-04
Validation auc = 0.684977
Epoch 130
Validation auc = 0.696244
Epoch 131
Validation auc = 0.705634
Epoch 132
Validation auc = 0.688263
Epoch 133
Validation auc = 0.680751
Epoch 134
Loss = 7.5504e-02, PNorm = 44.7071, GNorm = 0.8794, lr_0 = 8.0476e-04
Validation auc = 0.684038
Epoch 135
Validation auc = 0.695775
Epoch 136
Validation auc = 0.712207
Epoch 137
Validation auc = 0.708451
Epoch 138
Validation auc = 0.700469
Epoch 139
Loss = 1.6791e-01, PNorm = 44.7922, GNorm = 1.0759, lr_0 = 7.9788e-04
Validation auc = 0.701408
Epoch 140
Validation auc = 0.708451
Epoch 141
Validation auc = 0.711737
Epoch 142
Validation auc = 0.698592
Epoch 143
Validation auc = 0.682629
Epoch 144
Loss = 2.1366e-01, PNorm = 44.8739, GNorm = 1.1427, lr_0 = 7.9106e-04
Validation auc = 0.683568
Epoch 145
Validation auc = 0.683568
Epoch 146
Validation auc = 0.680282
Epoch 147
Validation auc = 0.669484
Epoch 148
Validation auc = 0.660094
Epoch 149
Loss = 1.7597e-01, PNorm = 44.9529, GNorm = 2.3501, lr_0 = 7.8430e-04
Validation auc = 0.661972
Epoch 150
Validation auc = 0.666197
Epoch 151
Validation auc = 0.654460
Epoch 152
Validation auc = 0.654930
Epoch 153
Validation auc = 0.658685
Epoch 154
Loss = 7.8221e-02, PNorm = 45.0335, GNorm = 0.8440, lr_0 = 7.7759e-04
Validation auc = 0.668075
Epoch 155
Validation auc = 0.664789
Epoch 156
Validation auc = 0.666667
Epoch 157
Validation auc = 0.666197
Epoch 158
Validation auc = 0.669014
Epoch 159
Loss = 8.2735e-02, PNorm = 45.1103, GNorm = 1.7760, lr_0 = 7.7095e-04
Validation auc = 0.683099
Epoch 160
Validation auc = 0.693427
Epoch 161
Validation auc = 0.687793
Epoch 162
Validation auc = 0.686854
Epoch 163
Validation auc = 0.687324
Epoch 164
Loss = 1.6215e-01, PNorm = 45.1860, GNorm = 4.2677, lr_0 = 7.6436e-04
Validation auc = 0.683099
Epoch 165
Validation auc = 0.666667
Epoch 166
Validation auc = 0.662911
Epoch 167
Validation auc = 0.662911
Epoch 168
Validation auc = 0.657746
Epoch 169
Loss = 1.5840e-01, PNorm = 45.2557, GNorm = 2.0262, lr_0 = 7.5782e-04
Validation auc = 0.665728
Epoch 170
Validation auc = 0.671831
Epoch 171
Validation auc = 0.688263
Epoch 172
Validation auc = 0.698122
Epoch 173
Validation auc = 0.698122
Epoch 174
Loss = 8.6383e-02, PNorm = 45.3202, GNorm = 0.4214, lr_0 = 7.5134e-04
Validation auc = 0.700000
Epoch 175
Validation auc = 0.689202
Epoch 176
Validation auc = 0.683568
Epoch 177
Validation auc = 0.687793
Epoch 178
Validation auc = 0.682160
Epoch 179
Loss = 1.1061e-01, PNorm = 45.3799, GNorm = 0.4204, lr_0 = 7.4492e-04
Validation auc = 0.676056
Epoch 180
Validation auc = 0.670892
Epoch 181
Validation auc = 0.665728
Epoch 182
Validation auc = 0.669953
Epoch 183
Validation auc = 0.679343
Epoch 184
Loss = 9.4903e-02, PNorm = 45.4368, GNorm = 2.6117, lr_0 = 7.3855e-04
Validation auc = 0.680282
Epoch 185
Validation auc = 0.676526
Epoch 186
Validation auc = 0.674178
Epoch 187
Validation auc = 0.680751
Epoch 188
Validation auc = 0.681221
Epoch 189
Loss = 7.8611e-02, PNorm = 45.4955, GNorm = 0.6771, lr_0 = 7.3224e-04
Validation auc = 0.679812
Epoch 190
Validation auc = 0.683099
Epoch 191
Validation auc = 0.692019
Epoch 192
Validation auc = 0.700939
Epoch 193
Validation auc = 0.694366
Epoch 194
Loss = 5.5354e-02, PNorm = 45.5552, GNorm = 0.7629, lr_0 = 7.2598e-04
Validation auc = 0.685446
Epoch 195
Validation auc = 0.682160
Epoch 196
Validation auc = 0.678404
Epoch 197
Validation auc = 0.673239
Epoch 198
Validation auc = 0.670423
Epoch 199
Loss = 9.4893e-02, PNorm = 45.6154, GNorm = 0.7186, lr_0 = 7.1977e-04
Validation auc = 0.677934
Epoch 200
Validation auc = 0.683099
Epoch 201
Validation auc = 0.687793
Epoch 202
Validation auc = 0.691549
Epoch 203
Validation auc = 0.693427
Epoch 204
Loss = 4.3552e-02, PNorm = 45.6730, GNorm = 1.8552, lr_0 = 7.1362e-04
Validation auc = 0.700469
Epoch 205
Validation auc = 0.709859
Epoch 206
Validation auc = 0.714554
Epoch 207
Validation auc = 0.722066
Epoch 208
Validation auc = 0.717840
Epoch 209
Loss = 2.0349e-02, PNorm = 45.7226, GNorm = 0.2098, lr_0 = 7.0752e-04
Validation auc = 0.714085
Epoch 210
Validation auc = 0.721596
Epoch 211
Validation auc = 0.730047
Epoch 212
Validation auc = 0.719718
Epoch 213
Validation auc = 0.700000
Epoch 214
Loss = 1.4160e-01, PNorm = 45.7735, GNorm = 1.4903, lr_0 = 7.0147e-04
Validation auc = 0.707512
Epoch 215
Validation auc = 0.713615
Epoch 216
Validation auc = 0.707512
Epoch 217
Validation auc = 0.700939
Epoch 218
Validation auc = 0.694836
Epoch 219
Loss = 4.2638e-02, PNorm = 45.8308, GNorm = 0.8078, lr_0 = 6.9548e-04
Validation auc = 0.700939
Epoch 220
Validation auc = 0.717840
Epoch 221
Validation auc = 0.714085
Epoch 222
Validation auc = 0.711268
Epoch 223
Validation auc = 0.713615
Epoch 224
Loss = 7.3135e-02, PNorm = 45.8913, GNorm = 1.4532, lr_0 = 6.8953e-04
Validation auc = 0.711268
Epoch 225
Validation auc = 0.700939
Epoch 226
Validation auc = 0.689202
Epoch 227
Validation auc = 0.692958
Epoch 228
Validation auc = 0.695775
Epoch 229
Loss = 8.0682e-02, PNorm = 45.9437, GNorm = 2.9357, lr_0 = 6.8364e-04
Validation auc = 0.696244
Epoch 230
Validation auc = 0.700000
Epoch 231
Validation auc = 0.701878
Epoch 232
Validation auc = 0.693897
Epoch 233
Validation auc = 0.692019
Epoch 234
Loss = 8.1829e-02, PNorm = 46.0058, GNorm = 0.7061, lr_0 = 6.7779e-04
Validation auc = 0.684977
Epoch 235
Validation auc = 0.682160
Epoch 236
Validation auc = 0.680282
Epoch 237
Validation auc = 0.685446
Epoch 238
Validation auc = 0.689671
Epoch 239
Loss = 4.0392e-02, PNorm = 46.0511, GNorm = 0.4537, lr_0 = 6.7200e-04
Validation auc = 0.693427
Epoch 240
Validation auc = 0.697183
Epoch 241
Validation auc = 0.700000
Epoch 242
Validation auc = 0.708451
Epoch 243
Validation auc = 0.710798
Epoch 244
Loss = 4.9633e-02, PNorm = 46.1024, GNorm = 1.4471, lr_0 = 6.6625e-04
Validation auc = 0.712676
Epoch 245
Validation auc = 0.709390
Epoch 246
Validation auc = 0.704225
Epoch 247
Validation auc = 0.702817
Epoch 248
Validation auc = 0.715023
Epoch 249
Loss = 4.2469e-02, PNorm = 46.1502, GNorm = 2.4780, lr_0 = 6.6056e-04
Validation auc = 0.711737
Epoch 250
Validation auc = 0.701878
Epoch 251
Validation auc = 0.693427
Epoch 252
Validation auc = 0.691080
Epoch 253
Validation auc = 0.690610
Epoch 254
Loss = 4.4941e-02, PNorm = 46.2022, GNorm = 1.1427, lr_0 = 6.5491e-04
Validation auc = 0.689671
Epoch 255
Validation auc = 0.684507
Epoch 256
Validation auc = 0.690141
Epoch 257
Validation auc = 0.697183
Epoch 258
Validation auc = 0.700469
Epoch 259
Loss = 1.6168e-02, PNorm = 46.2450, GNorm = 0.8403, lr_0 = 6.4931e-04
Validation auc = 0.707512
Epoch 260
Validation auc = 0.705634
Epoch 261
Validation auc = 0.707512
Epoch 262
Validation auc = 0.703286
Epoch 263
Validation auc = 0.693427
Epoch 264
Loss = 4.2876e-03, PNorm = 46.2806, GNorm = 0.1308, lr_0 = 6.4376e-04
Validation auc = 0.687324
Epoch 265
Validation auc = 0.688263
Epoch 266
Validation auc = 0.692488
Epoch 267
Validation auc = 0.692488
Epoch 268
Validation auc = 0.692488
Epoch 269
Loss = 9.4712e-02, PNorm = 46.3128, GNorm = 1.0867, lr_0 = 6.3826e-04
Validation auc = 0.700939
Epoch 270
Validation auc = 0.717371
Epoch 271
Validation auc = 0.719718
Epoch 272
Validation auc = 0.718779
Epoch 273
Validation auc = 0.719249
Epoch 274
Loss = 1.4152e-02, PNorm = 46.3421, GNorm = 0.6386, lr_0 = 6.3280e-04
Validation auc = 0.716901
Epoch 275
Validation auc = 0.713615
Epoch 276
Validation auc = 0.713146
Epoch 277
Validation auc = 0.715962
Epoch 278
Validation auc = 0.718779
Epoch 279
Loss = 4.9351e-02, PNorm = 46.3713, GNorm = 1.3551, lr_0 = 6.2739e-04
Validation auc = 0.717840
Epoch 280
Validation auc = 0.714554
Epoch 281
Validation auc = 0.704695
Epoch 282
Validation auc = 0.699061
Epoch 283
Validation auc = 0.689202
Epoch 284
Loss = 6.3503e-02, PNorm = 46.4042, GNorm = 0.9068, lr_0 = 6.2203e-04
Validation auc = 0.687324
Epoch 285
Validation auc = 0.697653
Epoch 286
Validation auc = 0.705634
Epoch 287
Validation auc = 0.706573
Epoch 288
Validation auc = 0.706573
Epoch 289
Loss = 7.3581e-03, PNorm = 46.4365, GNorm = 0.0783, lr_0 = 6.1671e-04
Validation auc = 0.709390
Epoch 290
Validation auc = 0.711268
Epoch 291
Validation auc = 0.715962
Epoch 292
Validation auc = 0.720188
Epoch 293
Validation auc = 0.713146
Epoch 294
Loss = 2.2615e-02, PNorm = 46.4647, GNorm = 0.9080, lr_0 = 6.1144e-04
Validation auc = 0.698122
Epoch 295
Validation auc = 0.693427
Epoch 296
Validation auc = 0.692019
Epoch 297
Validation auc = 0.692958
Epoch 298
Validation auc = 0.698592
Epoch 299
Loss = 2.0533e-02, PNorm = 46.4995, GNorm = 0.5006, lr_0 = 6.0621e-04
Validation auc = 0.708920
Model 0 best validation auc = 0.787324 on epoch 43
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.894040
Ensemble test auc = 0.894040
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.565728
Epoch 1
Validation auc = 0.529577
Epoch 2
Validation auc = 0.553521
Epoch 3
Validation auc = 0.531925
Epoch 4
Loss = 6.7906e-01, PNorm = 42.2837, GNorm = 1.0859, lr_0 = 6.5000e-04
Validation auc = 0.532394
Epoch 5
Validation auc = 0.545540
Epoch 6
Validation auc = 0.570892
Epoch 7
Validation auc = 0.604225
Epoch 8
Validation auc = 0.623474
Epoch 9
Loss = 6.8631e-01, PNorm = 42.3483, GNorm = 0.9020, lr_0 = 9.9743e-04
Validation auc = 0.628169
Epoch 10
Validation auc = 0.684977
Epoch 11
Validation auc = 0.660563
Epoch 12
Validation auc = 0.701878
Epoch 13
Validation auc = 0.691549
Epoch 14
Loss = 5.8714e-01, PNorm = 42.4425, GNorm = 0.9802, lr_0 = 9.8890e-04
Validation auc = 0.698592
Epoch 15
Validation auc = 0.691549
Epoch 16
Validation auc = 0.707042
Epoch 17
Validation auc = 0.704695
Epoch 18
Validation auc = 0.736620
Epoch 19
Loss = 5.4022e-01, PNorm = 42.5456, GNorm = 1.4057, lr_0 = 9.8045e-04
Validation auc = 0.733803
Epoch 20
Validation auc = 0.766667
Epoch 21
Validation auc = 0.735211
Epoch 22
Validation auc = 0.751643
Epoch 23
Validation auc = 0.744601
Epoch 24
Loss = 4.7743e-01, PNorm = 42.6337, GNorm = 1.6448, lr_0 = 9.7207e-04
Validation auc = 0.713146
Epoch 25
Validation auc = 0.740845
Epoch 26
Validation auc = 0.755399
Epoch 27
Validation auc = 0.732864
Epoch 28
Validation auc = 0.745540
Epoch 29
Loss = 5.1179e-01, PNorm = 42.7196, GNorm = 0.5572, lr_0 = 9.6376e-04
Validation auc = 0.761972
Epoch 30
Validation auc = 0.752113
Epoch 31
Validation auc = 0.749296
Epoch 32
Validation auc = 0.779343
Epoch 33
Validation auc = 0.756808
Epoch 34
Loss = 4.3432e-01, PNorm = 42.8050, GNorm = 1.3513, lr_0 = 9.5552e-04
Validation auc = 0.757746
Epoch 35
Validation auc = 0.774178
Epoch 36
Validation auc = 0.770892
Epoch 37
Validation auc = 0.758685
Epoch 38
Validation auc = 0.778404
Epoch 39
Loss = 4.2427e-01, PNorm = 42.8921, GNorm = 2.5892, lr_0 = 9.4735e-04
Validation auc = 0.778873
Epoch 40
Validation auc = 0.766197
Epoch 41
Validation auc = 0.779343
Epoch 42
Validation auc = 0.781690
Epoch 43
Validation auc = 0.767606
Epoch 44
Loss = 3.9233e-01, PNorm = 42.9779, GNorm = 0.7661, lr_0 = 9.3925e-04
Validation auc = 0.747418
Epoch 45
Validation auc = 0.761972
Epoch 46
Validation auc = 0.753052
Epoch 47
Validation auc = 0.740845
Epoch 48
Validation auc = 0.752582
Epoch 49
Loss = 4.2210e-01, PNorm = 43.0686, GNorm = 1.0022, lr_0 = 9.3122e-04
Validation auc = 0.763380
Epoch 50
Validation auc = 0.751174
Epoch 51
Validation auc = 0.749296
Epoch 52
Validation auc = 0.744601
Epoch 53
Validation auc = 0.744131
Epoch 54
Loss = 3.4958e-01, PNorm = 43.1506, GNorm = 2.1336, lr_0 = 9.2326e-04
Validation auc = 0.732394
Epoch 55
Validation auc = 0.728638
Epoch 56
Validation auc = 0.743192
Epoch 57
Validation auc = 0.721596
Epoch 58
Validation auc = 0.735211
Epoch 59
Loss = 3.7349e-01, PNorm = 43.2300, GNorm = 1.2744, lr_0 = 9.1537e-04
Validation auc = 0.753052
Epoch 60
Validation auc = 0.746009
Epoch 61
Validation auc = 0.751174
Epoch 62
Validation auc = 0.753991
Epoch 63
Validation auc = 0.748357
Epoch 64
Loss = 2.8514e-01, PNorm = 43.3258, GNorm = 0.8090, lr_0 = 9.0754e-04
Validation auc = 0.723005
Epoch 65
Validation auc = 0.743662
Epoch 66
Validation auc = 0.736150
Epoch 67
Validation auc = 0.751174
Epoch 68
Validation auc = 0.752582
Epoch 69
Loss = 2.7358e-01, PNorm = 43.4291, GNorm = 0.6810, lr_0 = 8.9978e-04
Validation auc = 0.739906
Epoch 70
Validation auc = 0.730516
Epoch 71
Validation auc = 0.731925
Epoch 72
Validation auc = 0.738967
Epoch 73
Validation auc = 0.723474
Epoch 74
Loss = 2.2175e-01, PNorm = 43.5340, GNorm = 1.1350, lr_0 = 8.9209e-04
Validation auc = 0.732394
Epoch 75
Validation auc = 0.727230
Epoch 76
Validation auc = 0.735681
Epoch 77
Validation auc = 0.712676
Epoch 78
Validation auc = 0.714554
Epoch 79
Loss = 2.1681e-01, PNorm = 43.6346, GNorm = 3.0219, lr_0 = 8.8447e-04
Validation auc = 0.702817
Epoch 80
Validation auc = 0.688732
Epoch 81
Validation auc = 0.718779
Epoch 82
Validation auc = 0.715493
Epoch 83
Validation auc = 0.723474
Epoch 84
Loss = 2.2765e-01, PNorm = 43.7356, GNorm = 5.3279, lr_0 = 8.7691e-04
Validation auc = 0.725352
Epoch 85
Validation auc = 0.697653
Epoch 86
Validation auc = 0.729108
Epoch 87
Validation auc = 0.723474
Epoch 88
Validation auc = 0.716901
Epoch 89
Loss = 3.1911e-01, PNorm = 43.8360, GNorm = 4.6098, lr_0 = 8.6941e-04
Validation auc = 0.741784
Epoch 90
Validation auc = 0.732394
Epoch 91
Validation auc = 0.717840
Epoch 92
Validation auc = 0.720657
Epoch 93
Validation auc = 0.725352
Epoch 94
Loss = 2.7350e-01, PNorm = 43.9272, GNorm = 2.0969, lr_0 = 8.6198e-04
Validation auc = 0.716432
Epoch 95
Validation auc = 0.696714
Epoch 96
Validation auc = 0.715493
Epoch 97
Validation auc = 0.719718
Epoch 98
Validation auc = 0.715493
Epoch 99
Loss = 1.5769e-01, PNorm = 44.0221, GNorm = 0.5792, lr_0 = 8.5461e-04
Validation auc = 0.706103
Epoch 100
Validation auc = 0.705634
Epoch 101
Validation auc = 0.712207
Epoch 102
Validation auc = 0.715493
Epoch 103
Validation auc = 0.701878
Epoch 104
Loss = 2.2784e-01, PNorm = 44.1223, GNorm = 1.7905, lr_0 = 8.4730e-04
Validation auc = 0.696244
Epoch 105
Validation auc = 0.703286
Epoch 106
Validation auc = 0.701878
Epoch 107
Validation auc = 0.692488
Epoch 108
Validation auc = 0.701878
Epoch 109
Loss = 1.2515e-01, PNorm = 44.2171, GNorm = 1.2934, lr_0 = 8.4006e-04
Validation auc = 0.702817
Epoch 110
Validation auc = 0.690610
Epoch 111
Validation auc = 0.697183
Epoch 112
Validation auc = 0.704695
Epoch 113
Validation auc = 0.689671
Epoch 114
Loss = 1.8391e-01, PNorm = 44.3062, GNorm = 4.5617, lr_0 = 8.3288e-04
Validation auc = 0.697183
Epoch 115
Validation auc = 0.716901
Epoch 116
Validation auc = 0.696714
Epoch 117
Validation auc = 0.685446
Epoch 118
Validation auc = 0.689202
Epoch 119
Loss = 1.1923e-01, PNorm = 44.3904, GNorm = 1.6212, lr_0 = 8.2576e-04
Validation auc = 0.701878
Epoch 120
Validation auc = 0.702347
Epoch 121
Validation auc = 0.705634
Epoch 122
Validation auc = 0.718779
Epoch 123
Validation auc = 0.719249
Epoch 124
Loss = 1.3017e-01, PNorm = 44.4728, GNorm = 0.7503, lr_0 = 8.1870e-04
Validation auc = 0.715493
Epoch 125
Validation auc = 0.705634
Epoch 126
Validation auc = 0.694836
Epoch 127
Validation auc = 0.701878
Epoch 128
Validation auc = 0.705634
Epoch 129
Loss = 1.5087e-01, PNorm = 44.5502, GNorm = 1.6542, lr_0 = 8.1170e-04
Validation auc = 0.702817
Epoch 130
Validation auc = 0.711737
Epoch 131
Validation auc = 0.707512
Epoch 132
Validation auc = 0.698592
Epoch 133
Validation auc = 0.704225
Epoch 134
Loss = 1.5443e-01, PNorm = 44.6197, GNorm = 1.5860, lr_0 = 8.0476e-04
Validation auc = 0.702347
Epoch 135
Validation auc = 0.692019
Epoch 136
Validation auc = 0.691549
Epoch 137
Validation auc = 0.695775
Epoch 138
Validation auc = 0.701878
Epoch 139
Loss = 7.5386e-02, PNorm = 44.6791, GNorm = 0.9380, lr_0 = 7.9788e-04
Validation auc = 0.705164
Epoch 140
Validation auc = 0.693427
Epoch 141
Validation auc = 0.693897
Epoch 142
Validation auc = 0.667136
Epoch 143
Validation auc = 0.667606
Epoch 144
Loss = 1.1286e-01, PNorm = 44.7369, GNorm = 2.0918, lr_0 = 7.9106e-04
Validation auc = 0.689202
Epoch 145
Validation auc = 0.693427
Epoch 146
Validation auc = 0.686854
Epoch 147
Validation auc = 0.693427
Epoch 148
Validation auc = 0.684038
Epoch 149
Loss = 1.3045e-01, PNorm = 44.7982, GNorm = 4.1634, lr_0 = 7.8430e-04
Validation auc = 0.681221
Epoch 150
Validation auc = 0.702817
Epoch 151
Validation auc = 0.689202
Epoch 152
Validation auc = 0.699061
Epoch 153
Validation auc = 0.715962
Epoch 154
Loss = 1.5322e-01, PNorm = 44.8621, GNorm = 2.4522, lr_0 = 7.7759e-04
Validation auc = 0.709859
Epoch 155
Validation auc = 0.699531
Epoch 156
Validation auc = 0.694836
Epoch 157
Validation auc = 0.696244
Epoch 158
Validation auc = 0.698122
Epoch 159
Loss = 1.4887e-01, PNorm = 44.9264, GNorm = 1.0707, lr_0 = 7.7095e-04
Validation auc = 0.692488
Epoch 160
Validation auc = 0.687324
Epoch 161
Validation auc = 0.678873
Epoch 162
Validation auc = 0.684977
Epoch 163
Validation auc = 0.689202
Epoch 164
Loss = 1.0041e-01, PNorm = 44.9818, GNorm = 0.9339, lr_0 = 7.6436e-04
Validation auc = 0.689671
Epoch 165
Validation auc = 0.696714
Epoch 166
Validation auc = 0.698122
Epoch 167
Validation auc = 0.710329
Epoch 168
Validation auc = 0.703756
Epoch 169
Loss = 9.4264e-02, PNorm = 45.0335, GNorm = 0.9102, lr_0 = 7.5782e-04
Validation auc = 0.688263
Epoch 170
Validation auc = 0.692019
Epoch 171
Validation auc = 0.701878
Epoch 172
Validation auc = 0.709390
Epoch 173
Validation auc = 0.703286
Epoch 174
Loss = 1.2681e-01, PNorm = 45.0769, GNorm = 3.7359, lr_0 = 7.5134e-04
Validation auc = 0.696244
Epoch 175
Validation auc = 0.701878
Epoch 176
Validation auc = 0.700000
Epoch 177
Validation auc = 0.697653
Epoch 178
Validation auc = 0.706103
Epoch 179
Loss = 6.0179e-02, PNorm = 45.1329, GNorm = 0.8295, lr_0 = 7.4492e-04
Validation auc = 0.706103
Epoch 180
Validation auc = 0.700469
Epoch 181
Validation auc = 0.690610
Epoch 182
Validation auc = 0.722535
Epoch 183
Validation auc = 0.711737
Epoch 184
Loss = 4.2524e-02, PNorm = 45.1977, GNorm = 3.2525, lr_0 = 7.3855e-04
Validation auc = 0.689671
Epoch 185
Validation auc = 0.697183
Epoch 186
Validation auc = 0.710798
Epoch 187
Validation auc = 0.714085
Epoch 188
Validation auc = 0.718310
Epoch 189
Loss = 5.8678e-02, PNorm = 45.2606, GNorm = 1.1123, lr_0 = 7.3224e-04
Validation auc = 0.726761
Epoch 190
Validation auc = 0.720188
Epoch 191
Validation auc = 0.721596
Epoch 192
Validation auc = 0.716432
Epoch 193
Validation auc = 0.707512
Epoch 194
Loss = 2.6729e-02, PNorm = 45.3219, GNorm = 1.0376, lr_0 = 7.2598e-04
Validation auc = 0.693427
Epoch 195
Validation auc = 0.695305
Epoch 196
Validation auc = 0.710798
Epoch 197
Validation auc = 0.718779
Epoch 198
Validation auc = 0.719718
Epoch 199
Loss = 8.5322e-02, PNorm = 45.3791, GNorm = 1.1350, lr_0 = 7.1977e-04
Validation auc = 0.729108
Epoch 200
Validation auc = 0.737559
Epoch 201
Validation auc = 0.732394
Epoch 202
Validation auc = 0.723474
Epoch 203
Validation auc = 0.721596
Epoch 204
Loss = 2.0916e-02, PNorm = 45.4325, GNorm = 1.4542, lr_0 = 7.1362e-04
Validation auc = 0.720657
Epoch 205
Validation auc = 0.734272
Epoch 206
Validation auc = 0.737089
Epoch 207
Validation auc = 0.734742
Epoch 208
Validation auc = 0.730986
Epoch 209
Loss = 9.4482e-02, PNorm = 45.4778, GNorm = 0.7260, lr_0 = 7.0752e-04
Validation auc = 0.729577
Epoch 210
Validation auc = 0.730986
Epoch 211
Validation auc = 0.719249
Epoch 212
Validation auc = 0.718310
Epoch 213
Validation auc = 0.715493
Epoch 214
Loss = 1.0341e-01, PNorm = 45.5179, GNorm = 1.7432, lr_0 = 7.0147e-04
Validation auc = 0.709390
Epoch 215
Validation auc = 0.702817
Epoch 216
Validation auc = 0.705164
Epoch 217
Validation auc = 0.714085
Epoch 218
Validation auc = 0.715962
Epoch 219
Loss = 3.2696e-02, PNorm = 45.5658, GNorm = 0.3864, lr_0 = 6.9548e-04
Validation auc = 0.714554
Epoch 220
Validation auc = 0.712207
Epoch 221
Validation auc = 0.713146
Epoch 222
Validation auc = 0.715023
Epoch 223
Validation auc = 0.726291
Epoch 224
Loss = 4.7833e-02, PNorm = 45.6168, GNorm = 2.3387, lr_0 = 6.8953e-04
Validation auc = 0.728638
Epoch 225
Validation auc = 0.718779
Epoch 226
Validation auc = 0.710329
Epoch 227
Validation auc = 0.721596
Epoch 228
Validation auc = 0.731455
Epoch 229
Loss = 4.1193e-02, PNorm = 45.6600, GNorm = 1.2018, lr_0 = 6.8364e-04
Validation auc = 0.733333
Epoch 230
Validation auc = 0.732394
Epoch 231
Validation auc = 0.730516
Epoch 232
Validation auc = 0.731455
Epoch 233
Validation auc = 0.733803
Epoch 234
Loss = 4.9941e-02, PNorm = 45.7044, GNorm = 2.8199, lr_0 = 6.7779e-04
Validation auc = 0.732864
Epoch 235
Validation auc = 0.729577
Epoch 236
Validation auc = 0.725822
Epoch 237
Validation auc = 0.724883
Epoch 238
Validation auc = 0.726761
Epoch 239
Loss = 5.7997e-02, PNorm = 45.7388, GNorm = 2.3920, lr_0 = 6.7200e-04
Validation auc = 0.723944
Epoch 240
Validation auc = 0.716432
Epoch 241
Validation auc = 0.715493
Epoch 242
Validation auc = 0.718779
Epoch 243
Validation auc = 0.720657
Epoch 244
Loss = 4.5931e-02, PNorm = 45.7717, GNorm = 1.3864, lr_0 = 6.6625e-04
Validation auc = 0.726761
Epoch 245
Validation auc = 0.728638
Epoch 246
Validation auc = 0.730516
Epoch 247
Validation auc = 0.733333
Epoch 248
Validation auc = 0.730516
Epoch 249
Loss = 3.4783e-02, PNorm = 45.8085, GNorm = 0.8349, lr_0 = 6.6056e-04
Validation auc = 0.728638
Epoch 250
Validation auc = 0.720657
Epoch 251
Validation auc = 0.713615
Epoch 252
Validation auc = 0.710798
Epoch 253
Validation auc = 0.710329
Epoch 254
Loss = 5.7735e-02, PNorm = 45.8477, GNorm = 0.8888, lr_0 = 6.5491e-04
Validation auc = 0.716901
Epoch 255
Validation auc = 0.717840
Epoch 256
Validation auc = 0.717371
Epoch 257
Validation auc = 0.721127
Epoch 258
Validation auc = 0.727700
Epoch 259
Loss = 1.5627e-02, PNorm = 45.8903, GNorm = 1.2318, lr_0 = 6.4931e-04
Validation auc = 0.740845
Epoch 260
Validation auc = 0.732394
Epoch 261
Validation auc = 0.717371
Epoch 262
Validation auc = 0.707981
Epoch 263
Validation auc = 0.709859
Epoch 264
Loss = 3.5239e-02, PNorm = 45.9359, GNorm = 2.1487, lr_0 = 6.4376e-04
Validation auc = 0.713615
Epoch 265
Validation auc = 0.712676
Epoch 266
Validation auc = 0.713146
Epoch 267
Validation auc = 0.721127
Epoch 268
Validation auc = 0.730047
Epoch 269
Loss = 6.1928e-02, PNorm = 45.9804, GNorm = 2.8309, lr_0 = 6.3826e-04
Validation auc = 0.730986
Epoch 270
Validation auc = 0.721596
Epoch 271
Validation auc = 0.717371
Epoch 272
Validation auc = 0.723005
Epoch 273
Validation auc = 0.728169
Epoch 274
Loss = 4.6562e-02, PNorm = 46.0313, GNorm = 0.8104, lr_0 = 6.3280e-04
Validation auc = 0.730516
Epoch 275
Validation auc = 0.729108
Epoch 276
Validation auc = 0.728169
Epoch 277
Validation auc = 0.731925
Epoch 278
Validation auc = 0.736620
Epoch 279
Loss = 1.5747e-02, PNorm = 46.0822, GNorm = 0.4787, lr_0 = 6.2739e-04
Validation auc = 0.740376
Epoch 280
Validation auc = 0.741784
Epoch 281
Validation auc = 0.736620
Epoch 282
Validation auc = 0.733803
Epoch 283
Validation auc = 0.731925
Epoch 284
Loss = 6.5600e-02, PNorm = 46.1233, GNorm = 1.1084, lr_0 = 6.2203e-04
Validation auc = 0.733333
Epoch 285
Validation auc = 0.723944
Epoch 286
Validation auc = 0.717840
Epoch 287
Validation auc = 0.710798
Epoch 288
Validation auc = 0.717371
Epoch 289
Loss = 4.5823e-02, PNorm = 46.1589, GNorm = 1.2439, lr_0 = 6.1671e-04
Validation auc = 0.730986
Epoch 290
Validation auc = 0.736150
Epoch 291
Validation auc = 0.732394
Epoch 292
Validation auc = 0.707981
Epoch 293
Validation auc = 0.712676
Epoch 294
Loss = 1.7809e-02, PNorm = 46.2020, GNorm = 2.2557, lr_0 = 6.1144e-04
Validation auc = 0.711737
Epoch 295
Validation auc = 0.712207
Epoch 296
Validation auc = 0.717840
Epoch 297
Validation auc = 0.724883
Epoch 298
Validation auc = 0.727230
Epoch 299
Loss = 3.2951e-02, PNorm = 46.2451, GNorm = 1.2182, lr_0 = 6.0621e-04
Validation auc = 0.724413
Model 0 best validation auc = 0.781690 on epoch 42
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.886815
Ensemble test auc = 0.886815
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.528638
Epoch 1
Validation auc = 0.534272
Epoch 2
Validation auc = 0.534272
Epoch 3
Validation auc = 0.538498
Epoch 4
Loss = 6.8842e-01, PNorm = 42.2838, GNorm = 1.0507, lr_0 = 6.5000e-04
Validation auc = 0.594836
Epoch 5
Validation auc = 0.602347
Epoch 6
Validation auc = 0.631455
Epoch 7
Validation auc = 0.646479
Epoch 8
Validation auc = 0.617840
Epoch 9
Loss = 6.4442e-01, PNorm = 42.3469, GNorm = 1.1210, lr_0 = 9.9743e-04
Validation auc = 0.624413
Epoch 10
Validation auc = 0.674178
Epoch 11
Validation auc = 0.669484
Epoch 12
Validation auc = 0.674178
Epoch 13
Validation auc = 0.667606
Epoch 14
Loss = 6.1831e-01, PNorm = 42.4456, GNorm = 1.1855, lr_0 = 9.8890e-04
Validation auc = 0.654460
Epoch 15
Validation auc = 0.674648
Epoch 16
Validation auc = 0.692958
Epoch 17
Validation auc = 0.709859
Epoch 18
Validation auc = 0.730047
Epoch 19
Loss = 5.3771e-01, PNorm = 42.5357, GNorm = 0.8518, lr_0 = 9.8045e-04
Validation auc = 0.700469
Epoch 20
Validation auc = 0.727700
Epoch 21
Validation auc = 0.720188
Epoch 22
Validation auc = 0.717371
Epoch 23
Validation auc = 0.742254
Epoch 24
Loss = 5.6403e-01, PNorm = 42.6241, GNorm = 1.4547, lr_0 = 9.7207e-04
Validation auc = 0.732864
Epoch 25
Validation auc = 0.716432
Epoch 26
Validation auc = 0.750235
Epoch 27
Validation auc = 0.733333
Epoch 28
Validation auc = 0.718310
Epoch 29
Loss = 5.3788e-01, PNorm = 42.7035, GNorm = 1.9440, lr_0 = 9.6376e-04
Validation auc = 0.741315
Epoch 30
Validation auc = 0.728638
Epoch 31
Validation auc = 0.730047
Epoch 32
Validation auc = 0.762911
Epoch 33
Validation auc = 0.753052
Epoch 34
Loss = 4.6768e-01, PNorm = 42.7907, GNorm = 1.0566, lr_0 = 9.5552e-04
Validation auc = 0.748826
Epoch 35
Validation auc = 0.756808
Epoch 36
Validation auc = 0.740376
Epoch 37
Validation auc = 0.755869
Epoch 38
Validation auc = 0.766197
Epoch 39
Loss = 4.2695e-01, PNorm = 42.8795, GNorm = 2.1312, lr_0 = 9.4735e-04
Validation auc = 0.764789
Epoch 40
Validation auc = 0.769484
Epoch 41
Validation auc = 0.763380
Epoch 42
Validation auc = 0.749765
Epoch 43
Validation auc = 0.766667
Epoch 44
Loss = 4.3620e-01, PNorm = 42.9622, GNorm = 0.8082, lr_0 = 9.3925e-04
Validation auc = 0.739437
Epoch 45
Validation auc = 0.744601
Epoch 46
Validation auc = 0.776056
Epoch 47
Validation auc = 0.769484
Epoch 48
Validation auc = 0.757277
Epoch 49
Loss = 4.7555e-01, PNorm = 43.0509, GNorm = 0.9914, lr_0 = 9.3122e-04
Validation auc = 0.765728
Epoch 50
Validation auc = 0.764789
Epoch 51
Validation auc = 0.754460
Epoch 52
Validation auc = 0.750704
Epoch 53
Validation auc = 0.755869
Epoch 54
Loss = 3.9374e-01, PNorm = 43.1410, GNorm = 0.8107, lr_0 = 9.2326e-04
Validation auc = 0.738028
Epoch 55
Validation auc = 0.738967
Epoch 56
Validation auc = 0.747887
Epoch 57
Validation auc = 0.727700
Epoch 58
Validation auc = 0.730986
Epoch 59
Loss = 4.2979e-01, PNorm = 43.2324, GNorm = 3.4594, lr_0 = 9.1537e-04
Validation auc = 0.723944
Epoch 60
Validation auc = 0.704695
Epoch 61
Validation auc = 0.736620
Epoch 62
Validation auc = 0.737089
Epoch 63
Validation auc = 0.730047
Epoch 64
Loss = 3.2925e-01, PNorm = 43.3388, GNorm = 1.3118, lr_0 = 9.0754e-04
Validation auc = 0.733333
Epoch 65
Validation auc = 0.728638
Epoch 66
Validation auc = 0.705634
Epoch 67
Validation auc = 0.718779
Epoch 68
Validation auc = 0.707042
Epoch 69
Loss = 2.9265e-01, PNorm = 43.4453, GNorm = 2.1562, lr_0 = 8.9978e-04
Validation auc = 0.724883
Epoch 70
Validation auc = 0.733803
Epoch 71
Validation auc = 0.721127
Epoch 72
Validation auc = 0.730047
Epoch 73
Validation auc = 0.711268
Epoch 74
Loss = 3.1722e-01, PNorm = 43.5566, GNorm = 1.9585, lr_0 = 8.9209e-04
Validation auc = 0.724883
Epoch 75
Validation auc = 0.715962
Epoch 76
Validation auc = 0.713146
Epoch 77
Validation auc = 0.716901
Epoch 78
Validation auc = 0.721127
Epoch 79
Loss = 2.4032e-01, PNorm = 43.6742, GNorm = 6.1786, lr_0 = 8.8447e-04
Validation auc = 0.706573
Epoch 80
Validation auc = 0.699061
Epoch 81
Validation auc = 0.677934
Epoch 82
Validation auc = 0.676995
Epoch 83
Validation auc = 0.688732
Epoch 84
Loss = 2.4082e-01, PNorm = 43.7837, GNorm = 1.2315, lr_0 = 8.7691e-04
Validation auc = 0.683099
Epoch 85
Validation auc = 0.700469
Epoch 86
Validation auc = 0.702817
Epoch 87
Validation auc = 0.696244
Epoch 88
Validation auc = 0.703286
Epoch 89
Loss = 1.4221e-01, PNorm = 43.8982, GNorm = 1.6792, lr_0 = 8.6941e-04
Validation auc = 0.707042
Epoch 90
Validation auc = 0.695305
Epoch 91
Validation auc = 0.689671
Epoch 92
Validation auc = 0.691549
Epoch 93
Validation auc = 0.679812
Epoch 94
Loss = 2.6559e-01, PNorm = 44.0105, GNorm = 2.9695, lr_0 = 8.6198e-04
Validation auc = 0.686854
Epoch 95
Validation auc = 0.675587
Epoch 96
Validation auc = 0.682160
Epoch 97
Validation auc = 0.699061
Epoch 98
Validation auc = 0.698592
Epoch 99
Loss = 1.6088e-01, PNorm = 44.1079, GNorm = 1.4950, lr_0 = 8.5461e-04
Validation auc = 0.705164
Epoch 100
Validation auc = 0.678873
Epoch 101
Validation auc = 0.685915
Epoch 102
Validation auc = 0.681221
Epoch 103
Validation auc = 0.677934
Epoch 104
Loss = 1.0118e-01, PNorm = 44.2102, GNorm = 1.2665, lr_0 = 8.4730e-04
Validation auc = 0.685446
Epoch 105
Validation auc = 0.690610
Epoch 106
Validation auc = 0.695775
Epoch 107
Validation auc = 0.703756
Epoch 108
Validation auc = 0.715962
Epoch 109
Loss = 2.2868e-01, PNorm = 44.2975, GNorm = 2.9613, lr_0 = 8.4006e-04
Validation auc = 0.681221
Epoch 110
Validation auc = 0.690610
Epoch 111
Validation auc = 0.709859
Epoch 112
Validation auc = 0.704225
Epoch 113
Validation auc = 0.703286
Epoch 114
Loss = 1.5081e-01, PNorm = 44.3851, GNorm = 2.9916, lr_0 = 8.3288e-04
Validation auc = 0.718310
Epoch 115
Validation auc = 0.722066
Epoch 116
Validation auc = 0.698122
Epoch 117
Validation auc = 0.683568
Epoch 118
Validation auc = 0.692019
Epoch 119
Loss = 2.5424e-01, PNorm = 44.4772, GNorm = 6.6472, lr_0 = 8.2576e-04
Validation auc = 0.689202
Epoch 120
Validation auc = 0.666197
Epoch 121
Validation auc = 0.688263
Epoch 122
Validation auc = 0.703756
Epoch 123
Validation auc = 0.713146
Epoch 124
Loss = 7.4076e-02, PNorm = 44.5632, GNorm = 0.3393, lr_0 = 8.1870e-04
Validation auc = 0.715493
Epoch 125
Validation auc = 0.722535
Epoch 126
Validation auc = 0.731925
Epoch 127
Validation auc = 0.729577
Epoch 128
Validation auc = 0.714085
Epoch 129
Loss = 1.0000e-01, PNorm = 44.6396, GNorm = 0.5433, lr_0 = 8.1170e-04
Validation auc = 0.699061
Epoch 130
Validation auc = 0.695775
Epoch 131
Validation auc = 0.695775
Epoch 132
Validation auc = 0.682629
Epoch 133
Validation auc = 0.683568
Epoch 134
Loss = 1.4832e-01, PNorm = 44.7040, GNorm = 0.6463, lr_0 = 8.0476e-04
Validation auc = 0.694366
Epoch 135
Validation auc = 0.705634
Epoch 136
Validation auc = 0.705634
Epoch 137
Validation auc = 0.701408
Epoch 138
Validation auc = 0.698592
Epoch 139
Loss = 1.2302e-01, PNorm = 44.7712, GNorm = 0.5116, lr_0 = 7.9788e-04
Validation auc = 0.702817
Epoch 140
Validation auc = 0.704695
Epoch 141
Validation auc = 0.697653
Epoch 142
Validation auc = 0.692958
Epoch 143
Validation auc = 0.702347
Epoch 144
Loss = 8.5719e-02, PNorm = 44.8356, GNorm = 3.6724, lr_0 = 7.9106e-04
Validation auc = 0.711268
Epoch 145
Validation auc = 0.681221
Epoch 146
Validation auc = 0.681690
Epoch 147
Validation auc = 0.691080
Epoch 148
Validation auc = 0.691549
Epoch 149
Loss = 9.5325e-02, PNorm = 44.9047, GNorm = 1.6231, lr_0 = 7.8430e-04
Validation auc = 0.678404
Epoch 150
Validation auc = 0.678873
Epoch 151
Validation auc = 0.690610
Epoch 152
Validation auc = 0.708920
Epoch 153
Validation auc = 0.712676
Epoch 154
Loss = 6.6695e-02, PNorm = 44.9699, GNorm = 0.3610, lr_0 = 7.7759e-04
Validation auc = 0.702347
Epoch 155
Validation auc = 0.704225
Epoch 156
Validation auc = 0.709390
Epoch 157
Validation auc = 0.719718
Epoch 158
Validation auc = 0.710329
Epoch 159
Loss = 6.2144e-02, PNorm = 45.0345, GNorm = 0.6226, lr_0 = 7.7095e-04
Validation auc = 0.687324
Epoch 160
Validation auc = 0.675117
Epoch 161
Validation auc = 0.678404
Epoch 162
Validation auc = 0.679343
Epoch 163
Validation auc = 0.674178
Epoch 164
Loss = 1.0864e-01, PNorm = 45.0976, GNorm = 1.0745, lr_0 = 7.6436e-04
Validation auc = 0.663850
Epoch 165
Validation auc = 0.668075
Epoch 166
Validation auc = 0.671831
Epoch 167
Validation auc = 0.676995
Epoch 168
Validation auc = 0.682629
Epoch 169
Loss = 1.0040e-01, PNorm = 45.1544, GNorm = 0.5432, lr_0 = 7.5782e-04
Validation auc = 0.688732
Epoch 170
Validation auc = 0.687793
Epoch 171
Validation auc = 0.692019
Epoch 172
Validation auc = 0.696714
Epoch 173
Validation auc = 0.695305
Epoch 174
Loss = 4.5190e-02, PNorm = 45.2083, GNorm = 0.7497, lr_0 = 7.5134e-04
Validation auc = 0.699061
Epoch 175
Validation auc = 0.695305
Epoch 176
Validation auc = 0.689671
Epoch 177
Validation auc = 0.687324
Epoch 178
Validation auc = 0.696244
Epoch 179
Loss = 6.9691e-02, PNorm = 45.2595, GNorm = 0.4756, lr_0 = 7.4492e-04
Validation auc = 0.700469
Epoch 180
Validation auc = 0.700939
Epoch 181
Validation auc = 0.696244
Epoch 182
Validation auc = 0.695775
Epoch 183
Validation auc = 0.695305
Epoch 184
Loss = 4.1310e-02, PNorm = 45.3041, GNorm = 0.9581, lr_0 = 7.3855e-04
Validation auc = 0.687793
Epoch 185
Validation auc = 0.687793
Epoch 186
Validation auc = 0.689202
Epoch 187
Validation auc = 0.687324
Epoch 188
Validation auc = 0.688263
Epoch 189
Loss = 4.2119e-02, PNorm = 45.3583, GNorm = 1.3590, lr_0 = 7.3224e-04
Validation auc = 0.687324
Epoch 190
Validation auc = 0.715493
Epoch 191
Validation auc = 0.709390
Epoch 192
Validation auc = 0.693427
Epoch 193
Validation auc = 0.690141
Epoch 194
Loss = 1.8289e-01, PNorm = 45.4120, GNorm = 0.8274, lr_0 = 7.2598e-04
Validation auc = 0.712207
Epoch 195
Validation auc = 0.708920
Epoch 196
Validation auc = 0.692019
Epoch 197
Validation auc = 0.694836
Epoch 198
Validation auc = 0.699531
Epoch 199
Loss = 6.8709e-02, PNorm = 45.4730, GNorm = 0.7922, lr_0 = 7.1977e-04
Validation auc = 0.700469
Epoch 200
Validation auc = 0.697183
Epoch 201
Validation auc = 0.699061
Epoch 202
Validation auc = 0.701878
Epoch 203
Validation auc = 0.704695
Epoch 204
Loss = 4.4909e-02, PNorm = 45.5348, GNorm = 0.9781, lr_0 = 7.1362e-04
Validation auc = 0.707042
Epoch 205
Validation auc = 0.705634
Epoch 206
Validation auc = 0.703756
Epoch 207
Validation auc = 0.704225
Epoch 208
Validation auc = 0.707042
Epoch 209
Loss = 5.9051e-02, PNorm = 45.5962, GNorm = 1.4730, lr_0 = 7.0752e-04
Validation auc = 0.707042
Epoch 210
Validation auc = 0.693897
Epoch 211
Validation auc = 0.689202
Epoch 212
Validation auc = 0.696244
Epoch 213
Validation auc = 0.699531
Epoch 214
Loss = 1.1079e-01, PNorm = 45.6462, GNorm = 0.7438, lr_0 = 7.0147e-04
Validation auc = 0.700469
Epoch 215
Validation auc = 0.702817
Epoch 216
Validation auc = 0.703286
Epoch 217
Validation auc = 0.703756
Epoch 218
Validation auc = 0.702817
Epoch 219
Loss = 4.7184e-02, PNorm = 45.7013, GNorm = 1.1337, lr_0 = 6.9548e-04
Validation auc = 0.713146
Epoch 220
Validation auc = 0.717371
Epoch 221
Validation auc = 0.701408
Epoch 222
Validation auc = 0.711268
Epoch 223
Validation auc = 0.728169
Epoch 224
Loss = 9.0465e-02, PNorm = 45.7392, GNorm = 4.0561, lr_0 = 6.8953e-04
Validation auc = 0.730047
Epoch 225
Validation auc = 0.730516
Epoch 226
Validation auc = 0.734272
Epoch 227
Validation auc = 0.732864
Epoch 228
Validation auc = 0.732394
Epoch 229
Loss = 5.1899e-02, PNorm = 45.7866, GNorm = 1.0909, lr_0 = 6.8364e-04
Validation auc = 0.729108
Epoch 230
Validation auc = 0.715962
Epoch 231
Validation auc = 0.708920
Epoch 232
Validation auc = 0.700469
Epoch 233
Validation auc = 0.688263
Epoch 234
Loss = 8.4858e-02, PNorm = 45.8325, GNorm = 0.8168, lr_0 = 6.7779e-04
Validation auc = 0.690141
Epoch 235
Validation auc = 0.683568
Epoch 236
Validation auc = 0.678873
Epoch 237
Validation auc = 0.674648
Epoch 238
Validation auc = 0.676995
Epoch 239
Loss = 2.4349e-02, PNorm = 45.8758, GNorm = 0.2351, lr_0 = 6.7200e-04
Validation auc = 0.684507
Epoch 240
Validation auc = 0.702347
Epoch 241
Validation auc = 0.718779
Epoch 242
Validation auc = 0.715023
Epoch 243
Validation auc = 0.722535
Epoch 244
Loss = 4.6667e-02, PNorm = 45.9040, GNorm = 1.2330, lr_0 = 6.6625e-04
Validation auc = 0.724413
Epoch 245
Validation auc = 0.722066
Epoch 246
Validation auc = 0.718310
Epoch 247
Validation auc = 0.712207
Epoch 248
Validation auc = 0.708451
Epoch 249
Loss = 7.9837e-02, PNorm = 45.9407, GNorm = 1.7520, lr_0 = 6.6056e-04
Validation auc = 0.709390
Epoch 250
Validation auc = 0.707042
Epoch 251
Validation auc = 0.705164
Epoch 252
Validation auc = 0.701878
Epoch 253
Validation auc = 0.700000
Epoch 254
Loss = 2.1946e-02, PNorm = 45.9831, GNorm = 1.0486, lr_0 = 6.5491e-04
Validation auc = 0.699531
Epoch 255
Validation auc = 0.698592
Epoch 256
Validation auc = 0.695775
Epoch 257
Validation auc = 0.687324
Epoch 258
Validation auc = 0.678873
Epoch 259
Loss = 2.4999e-02, PNorm = 46.0193, GNorm = 0.1768, lr_0 = 6.4931e-04
Validation auc = 0.672770
Epoch 260
Validation auc = 0.671362
Epoch 261
Validation auc = 0.675117
Epoch 262
Validation auc = 0.681690
Epoch 263
Validation auc = 0.680282
Epoch 264
Loss = 7.8685e-02, PNorm = 46.0542, GNorm = 0.5664, lr_0 = 6.4376e-04
Validation auc = 0.683568
Epoch 265
Validation auc = 0.690610
Epoch 266
Validation auc = 0.696714
Epoch 267
Validation auc = 0.700469
Epoch 268
Validation auc = 0.695305
Epoch 269
Loss = 3.0217e-02, PNorm = 46.0853, GNorm = 0.9974, lr_0 = 6.3826e-04
Validation auc = 0.693427
Epoch 270
Validation auc = 0.691080
Epoch 271
Validation auc = 0.690141
Epoch 272
Validation auc = 0.687324
Epoch 273
Validation auc = 0.681690
Epoch 274
Loss = 6.3166e-03, PNorm = 46.1148, GNorm = 0.1978, lr_0 = 6.3280e-04
Validation auc = 0.683099
Epoch 275
Validation auc = 0.692019
Epoch 276
Validation auc = 0.691549
Epoch 277
Validation auc = 0.682160
Epoch 278
Validation auc = 0.672770
Epoch 279
Loss = 5.0658e-02, PNorm = 46.1413, GNorm = 1.5607, lr_0 = 6.2739e-04
Validation auc = 0.670423
Epoch 280
Validation auc = 0.678873
Epoch 281
Validation auc = 0.691549
Epoch 282
Validation auc = 0.693897
Epoch 283
Validation auc = 0.692958
Epoch 284
Loss = 1.6879e-02, PNorm = 46.1732, GNorm = 0.3387, lr_0 = 6.2203e-04
Validation auc = 0.688263
Epoch 285
Validation auc = 0.685915
Epoch 286
Validation auc = 0.686385
Epoch 287
Validation auc = 0.690141
Epoch 288
Validation auc = 0.693427
Epoch 289
Loss = 2.5990e-02, PNorm = 46.2041, GNorm = 1.0997, lr_0 = 6.1671e-04
Validation auc = 0.696244
Epoch 290
Validation auc = 0.698122
Epoch 291
Validation auc = 0.697183
Epoch 292
Validation auc = 0.694366
Epoch 293
Validation auc = 0.701878
Epoch 294
Loss = 7.7696e-03, PNorm = 46.2316, GNorm = 0.2850, lr_0 = 6.1144e-04
Validation auc = 0.707512
Epoch 295
Validation auc = 0.709390
Epoch 296
Validation auc = 0.710798
Epoch 297
Validation auc = 0.708920
Epoch 298
Validation auc = 0.703756
Epoch 299
Loss = 5.7358e-02, PNorm = 46.2547, GNorm = 0.7115, lr_0 = 6.0621e-04
Validation auc = 0.704225
Model 0 best validation auc = 0.776056 on epoch 46
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.889223
Ensemble test auc = 0.889223
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.540376
Epoch 1
Validation auc = 0.523474
Epoch 2
Validation auc = 0.541315
Epoch 3
Validation auc = 0.532864
Epoch 4
Loss = 6.7919e-01, PNorm = 42.2862, GNorm = 1.2294, lr_0 = 6.5000e-04
Validation auc = 0.538967
Epoch 5
Validation auc = 0.606103
Epoch 6
Validation auc = 0.607512
Epoch 7
Validation auc = 0.602347
Epoch 8
Validation auc = 0.641315
Epoch 9
Loss = 6.6359e-01, PNorm = 42.3497, GNorm = 0.8798, lr_0 = 9.9743e-04
Validation auc = 0.631925
Epoch 10
Validation auc = 0.665258
Epoch 11
Validation auc = 0.665258
Epoch 12
Validation auc = 0.697183
Epoch 13
Validation auc = 0.707042
Epoch 14
Loss = 5.8430e-01, PNorm = 42.4450, GNorm = 0.8011, lr_0 = 9.8890e-04
Validation auc = 0.685446
Epoch 15
Validation auc = 0.695305
Epoch 16
Validation auc = 0.683099
Epoch 17
Validation auc = 0.690141
Epoch 18
Validation auc = 0.711268
Epoch 19
Loss = 5.7801e-01, PNorm = 42.5425, GNorm = 2.2368, lr_0 = 9.8045e-04
Validation auc = 0.694836
Epoch 20
Validation auc = 0.727230
Epoch 21
Validation auc = 0.714085
Epoch 22
Validation auc = 0.731925
Epoch 23
Validation auc = 0.733803
Epoch 24
Loss = 5.2261e-01, PNorm = 42.6222, GNorm = 1.2543, lr_0 = 9.7207e-04
Validation auc = 0.757277
Epoch 25
Validation auc = 0.741315
Epoch 26
Validation auc = 0.739906
Epoch 27
Validation auc = 0.770892
Epoch 28
Validation auc = 0.736150
Epoch 29
Loss = 5.4009e-01, PNorm = 42.7052, GNorm = 3.8161, lr_0 = 9.6376e-04
Validation auc = 0.774648
Epoch 30
Validation auc = 0.773239
Epoch 31
Validation auc = 0.743662
Epoch 32
Validation auc = 0.756808
Epoch 33
Validation auc = 0.775117
Epoch 34
Loss = 4.9365e-01, PNorm = 42.7737, GNorm = 2.5128, lr_0 = 9.5552e-04
Validation auc = 0.765728
Epoch 35
Validation auc = 0.734272
Epoch 36
Validation auc = 0.763380
Epoch 37
Validation auc = 0.775587
Epoch 38
Validation auc = 0.769953
Epoch 39
Loss = 5.1005e-01, PNorm = 42.8511, GNorm = 0.8806, lr_0 = 9.4735e-04
Validation auc = 0.767136
Epoch 40
Validation auc = 0.759624
Epoch 41
Validation auc = 0.779812
Epoch 42
Validation auc = 0.779343
Epoch 43
Validation auc = 0.754930
Epoch 44
Loss = 5.2616e-01, PNorm = 42.9373, GNorm = 2.4728, lr_0 = 9.3925e-04
Validation auc = 0.770892
Epoch 45
Validation auc = 0.781221
Epoch 46
Validation auc = 0.774178
Epoch 47
Validation auc = 0.761033
Epoch 48
Validation auc = 0.764319
Epoch 49
Loss = 3.3365e-01, PNorm = 43.0197, GNorm = 0.7469, lr_0 = 9.3122e-04
Validation auc = 0.772770
Epoch 50
Validation auc = 0.766197
Epoch 51
Validation auc = 0.768545
Epoch 52
Validation auc = 0.765728
Epoch 53
Validation auc = 0.769014
Epoch 54
Loss = 4.1154e-01, PNorm = 43.1196, GNorm = 1.0537, lr_0 = 9.2326e-04
Validation auc = 0.768075
Epoch 55
Validation auc = 0.750704
Epoch 56
Validation auc = 0.740376
Epoch 57
Validation auc = 0.743192
Epoch 58
Validation auc = 0.749296
Epoch 59
Loss = 2.4158e-01, PNorm = 43.2081, GNorm = 1.7093, lr_0 = 9.1537e-04
Validation auc = 0.751643
Epoch 60
Validation auc = 0.752582
Epoch 61
Validation auc = 0.754930
Epoch 62
Validation auc = 0.761033
Epoch 63
Validation auc = 0.760563
Epoch 64
Loss = 3.6630e-01, PNorm = 43.3163, GNorm = 1.1390, lr_0 = 9.0754e-04
Validation auc = 0.754460
Epoch 65
Validation auc = 0.730986
Epoch 66
Validation auc = 0.711737
Epoch 67
Validation auc = 0.729108
Epoch 68
Validation auc = 0.723474
Epoch 69
Loss = 3.3767e-01, PNorm = 43.4167, GNorm = 1.1598, lr_0 = 8.9978e-04
Validation auc = 0.742723
Epoch 70
Validation auc = 0.734272
Epoch 71
Validation auc = 0.731455
Epoch 72
Validation auc = 0.733333
Epoch 73
Validation auc = 0.718779
Epoch 74
Loss = 3.2820e-01, PNorm = 43.5222, GNorm = 2.7204, lr_0 = 8.9209e-04
Validation auc = 0.719249
Epoch 75
Validation auc = 0.731455
Epoch 76
Validation auc = 0.717371
Epoch 77
Validation auc = 0.721127
Epoch 78
Validation auc = 0.729577
Epoch 79
Loss = 2.5059e-01, PNorm = 43.6340, GNorm = 1.6008, lr_0 = 8.8447e-04
Validation auc = 0.718779
Epoch 80
Validation auc = 0.715962
Epoch 81
Validation auc = 0.715023
Epoch 82
Validation auc = 0.708920
Epoch 83
Validation auc = 0.714085
Epoch 84
Loss = 2.1916e-01, PNorm = 43.7402, GNorm = 1.4077, lr_0 = 8.7691e-04
Validation auc = 0.714085
Epoch 85
Validation auc = 0.701878
Epoch 86
Validation auc = 0.712676
Epoch 87
Validation auc = 0.707981
Epoch 88
Validation auc = 0.706103
Epoch 89
Loss = 2.9259e-01, PNorm = 43.8463, GNorm = 1.3312, lr_0 = 8.6941e-04
Validation auc = 0.716432
Epoch 90
Validation auc = 0.723474
Epoch 91
Validation auc = 0.719249
Epoch 92
Validation auc = 0.712207
Epoch 93
Validation auc = 0.717840
Epoch 94
Loss = 2.6172e-01, PNorm = 43.9449, GNorm = 1.4170, lr_0 = 8.6198e-04
Validation auc = 0.712207
Epoch 95
Validation auc = 0.715493
Epoch 96
Validation auc = 0.712207
Epoch 97
Validation auc = 0.692958
Epoch 98
Validation auc = 0.703286
Epoch 99
Loss = 2.0354e-01, PNorm = 44.0458, GNorm = 6.4161, lr_0 = 8.5461e-04
Validation auc = 0.678873
Epoch 100
Validation auc = 0.674648
Epoch 101
Validation auc = 0.700939
Epoch 102
Validation auc = 0.704695
Epoch 103
Validation auc = 0.701408
Epoch 104
Loss = 2.1397e-01, PNorm = 44.1448, GNorm = 3.4538, lr_0 = 8.4730e-04
Validation auc = 0.705634
Epoch 105
Validation auc = 0.713615
Epoch 106
Validation auc = 0.696714
Epoch 107
Validation auc = 0.692958
Epoch 108
Validation auc = 0.697653
Epoch 109
Loss = 1.4487e-01, PNorm = 44.2324, GNorm = 3.0740, lr_0 = 8.4006e-04
Validation auc = 0.686854
Epoch 110
Validation auc = 0.693427
Epoch 111
Validation auc = 0.698592
Epoch 112
Validation auc = 0.695775
Epoch 113
Validation auc = 0.689671
Epoch 114
Loss = 1.1980e-01, PNorm = 44.3186, GNorm = 1.2305, lr_0 = 8.3288e-04
Validation auc = 0.690141
Epoch 115
Validation auc = 0.689671
Epoch 116
Validation auc = 0.694836
Epoch 117
Validation auc = 0.707981
Epoch 118
Validation auc = 0.700000
Epoch 119
Loss = 1.1224e-01, PNorm = 44.4083, GNorm = 2.1799, lr_0 = 8.2576e-04
Validation auc = 0.696244
Epoch 120
Validation auc = 0.718310
Epoch 121
Validation auc = 0.693427
Epoch 122
Validation auc = 0.691080
Epoch 123
Validation auc = 0.703756
Epoch 124
Loss = 1.1998e-01, PNorm = 44.4823, GNorm = 0.7973, lr_0 = 8.1870e-04
Validation auc = 0.706573
Epoch 125
Validation auc = 0.714085
Epoch 126
Validation auc = 0.723474
Epoch 127
Validation auc = 0.723005
Epoch 128
Validation auc = 0.704695
Epoch 129
Loss = 1.5501e-01, PNorm = 44.5594, GNorm = 2.1648, lr_0 = 8.1170e-04
Validation auc = 0.697653
Epoch 130
Validation auc = 0.689671
Epoch 131
Validation auc = 0.681690
Epoch 132
Validation auc = 0.687324
Epoch 133
Validation auc = 0.698122
Epoch 134
Loss = 1.2633e-01, PNorm = 44.6405, GNorm = 2.2629, lr_0 = 8.0476e-04
Validation auc = 0.704695
Epoch 135
Validation auc = 0.711268
Epoch 136
Validation auc = 0.709859
Epoch 137
Validation auc = 0.713146
Epoch 138
Validation auc = 0.724413
Epoch 139
Loss = 1.4311e-01, PNorm = 44.7152, GNorm = 2.2889, lr_0 = 7.9788e-04
Validation auc = 0.717840
Epoch 140
Validation auc = 0.710329
Epoch 141
Validation auc = 0.707981
Epoch 142
Validation auc = 0.717840
Epoch 143
Validation auc = 0.717371
Epoch 144
Loss = 2.3434e-01, PNorm = 44.7839, GNorm = 1.3582, lr_0 = 7.9106e-04
Validation auc = 0.700000
Epoch 145
Validation auc = 0.699061
Epoch 146
Validation auc = 0.698592
Epoch 147
Validation auc = 0.697183
Epoch 148
Validation auc = 0.688732
Epoch 149
Loss = 1.7395e-01, PNorm = 44.8558, GNorm = 1.4734, lr_0 = 7.8430e-04
Validation auc = 0.682629
Epoch 150
Validation auc = 0.682160
Epoch 151
Validation auc = 0.678404
Epoch 152
Validation auc = 0.679812
Epoch 153
Validation auc = 0.681690
Epoch 154
Loss = 1.2466e-01, PNorm = 44.9156, GNorm = 2.8687, lr_0 = 7.7759e-04
Validation auc = 0.686385
Epoch 155
Validation auc = 0.695305
Epoch 156
Validation auc = 0.712207
Epoch 157
Validation auc = 0.707981
Epoch 158
Validation auc = 0.694366
Epoch 159
Loss = 6.2810e-02, PNorm = 44.9766, GNorm = 1.3835, lr_0 = 7.7095e-04
Validation auc = 0.684507
Epoch 160
Validation auc = 0.687793
Epoch 161
Validation auc = 0.691549
Epoch 162
Validation auc = 0.694836
Epoch 163
Validation auc = 0.695775
Epoch 164
Loss = 1.0470e-01, PNorm = 45.0472, GNorm = 1.2852, lr_0 = 7.6436e-04
Validation auc = 0.698592
Epoch 165
Validation auc = 0.704225
Epoch 166
Validation auc = 0.707981
Epoch 167
Validation auc = 0.707042
Epoch 168
Validation auc = 0.700469
Epoch 169
Loss = 3.5643e-02, PNorm = 45.1184, GNorm = 0.5644, lr_0 = 7.5782e-04
Validation auc = 0.700469
Epoch 170
Validation auc = 0.706103
Epoch 171
Validation auc = 0.700000
Epoch 172
Validation auc = 0.700939
Epoch 173
Validation auc = 0.709390
Epoch 174
Loss = 1.1322e-01, PNorm = 45.1705, GNorm = 1.0938, lr_0 = 7.5134e-04
Validation auc = 0.703286
Epoch 175
Validation auc = 0.700469
Epoch 176
Validation auc = 0.713615
Epoch 177
Validation auc = 0.708451
Epoch 178
Validation auc = 0.701878
Epoch 179
Loss = 5.5712e-02, PNorm = 45.2229, GNorm = 1.3737, lr_0 = 7.4492e-04
Validation auc = 0.689202
Epoch 180
Validation auc = 0.685915
Epoch 181
Validation auc = 0.690141
Epoch 182
Validation auc = 0.689671
Epoch 183
Validation auc = 0.671362
Epoch 184
Loss = 9.4069e-02, PNorm = 45.2904, GNorm = 1.6399, lr_0 = 7.3855e-04
Validation auc = 0.663850
Epoch 185
Validation auc = 0.673239
Epoch 186
Validation auc = 0.698592
Epoch 187
Validation auc = 0.690141
Epoch 188
Validation auc = 0.676526
Epoch 189
Loss = 6.8069e-02, PNorm = 45.3462, GNorm = 1.4662, lr_0 = 7.3224e-04
Validation auc = 0.676056
Epoch 190
Validation auc = 0.674648
Epoch 191
Validation auc = 0.680282
Epoch 192
Validation auc = 0.680282
Epoch 193
Validation auc = 0.674648
Epoch 194
Loss = 1.3220e-01, PNorm = 45.4032, GNorm = 2.9644, lr_0 = 7.2598e-04
Validation auc = 0.679812
Epoch 195
Validation auc = 0.683099
Epoch 196
Validation auc = 0.684977
Epoch 197
Validation auc = 0.682160
Epoch 198
Validation auc = 0.682160
Epoch 199
Loss = 1.3472e-01, PNorm = 45.4643, GNorm = 3.1069, lr_0 = 7.1977e-04
Validation auc = 0.694836
Epoch 200
Validation auc = 0.703286
Epoch 201
Validation auc = 0.709859
Epoch 202
Validation auc = 0.713615
Epoch 203
Validation auc = 0.709859
Epoch 204
Loss = 5.3750e-02, PNorm = 45.5231, GNorm = 1.8518, lr_0 = 7.1362e-04
Validation auc = 0.703286
Epoch 205
Validation auc = 0.698122
Epoch 206
Validation auc = 0.693427
Epoch 207
Validation auc = 0.692958
Epoch 208
Validation auc = 0.685915
Epoch 209
Loss = 2.5668e-02, PNorm = 45.5784, GNorm = 0.4271, lr_0 = 7.0752e-04
Validation auc = 0.677465
Epoch 210
Validation auc = 0.672300
Epoch 211
Validation auc = 0.682629
Epoch 212
Validation auc = 0.687324
Epoch 213
Validation auc = 0.688263
Epoch 214
Loss = 1.6486e-02, PNorm = 45.6253, GNorm = 0.3588, lr_0 = 7.0147e-04
Validation auc = 0.688263
Epoch 215
Validation auc = 0.698122
Epoch 216
Validation auc = 0.712676
Epoch 217
Validation auc = 0.708451
Epoch 218
Validation auc = 0.707512
Epoch 219
Loss = 4.2109e-02, PNorm = 45.6635, GNorm = 1.4547, lr_0 = 6.9548e-04
Validation auc = 0.707512
Epoch 220
Validation auc = 0.702817
Epoch 221
Validation auc = 0.697183
Epoch 222
Validation auc = 0.696244
Epoch 223
Validation auc = 0.692019
Epoch 224
Loss = 6.5316e-02, PNorm = 45.7068, GNorm = 1.8946, lr_0 = 6.8953e-04
Validation auc = 0.691080
Epoch 225
Validation auc = 0.692958
Epoch 226
Validation auc = 0.693897
Epoch 227
Validation auc = 0.679812
Epoch 228
Validation auc = 0.674648
Epoch 229
Loss = 1.1307e-01, PNorm = 45.7503, GNorm = 1.2333, lr_0 = 6.8364e-04
Validation auc = 0.689202
Epoch 230
Validation auc = 0.709859
Epoch 231
Validation auc = 0.705634
Epoch 232
Validation auc = 0.698592
Epoch 233
Validation auc = 0.700939
Epoch 234
Loss = 9.5474e-02, PNorm = 45.7974, GNorm = 1.1552, lr_0 = 6.7779e-04
Validation auc = 0.703756
Epoch 235
Validation auc = 0.697183
Epoch 236
Validation auc = 0.698122
Epoch 237
Validation auc = 0.697183
Epoch 238
Validation auc = 0.707981
Epoch 239
Loss = 7.0313e-02, PNorm = 45.8437, GNorm = 0.8952, lr_0 = 6.7200e-04
Validation auc = 0.715023
Epoch 240
Validation auc = 0.711737
Epoch 241
Validation auc = 0.705634
Epoch 242
Validation auc = 0.703756
Epoch 243
Validation auc = 0.704695
Epoch 244
Loss = 6.7406e-02, PNorm = 45.8872, GNorm = 3.1486, lr_0 = 6.6625e-04
Validation auc = 0.707981
Epoch 245
Validation auc = 0.709859
Epoch 246
Validation auc = 0.713615
Epoch 247
Validation auc = 0.708920
Epoch 248
Validation auc = 0.709859
Epoch 249
Loss = 5.1260e-02, PNorm = 45.9274, GNorm = 1.4290, lr_0 = 6.6056e-04
Validation auc = 0.710798
Epoch 250
Validation auc = 0.712676
Epoch 251
Validation auc = 0.710329
Epoch 252
Validation auc = 0.707512
Epoch 253
Validation auc = 0.701878
Epoch 254
Loss = 2.2494e-02, PNorm = 45.9675, GNorm = 0.9538, lr_0 = 6.5491e-04
Validation auc = 0.696244
Epoch 255
Validation auc = 0.704695
Epoch 256
Validation auc = 0.712676
Epoch 257
Validation auc = 0.716432
Epoch 258
Validation auc = 0.716901
Epoch 259
Loss = 1.8683e-02, PNorm = 46.0068, GNorm = 0.4646, lr_0 = 6.4931e-04
Validation auc = 0.723005
Epoch 260
Validation auc = 0.722066
Epoch 261
Validation auc = 0.715023
Epoch 262
Validation auc = 0.708920
Epoch 263
Validation auc = 0.706573
Epoch 264
Loss = 1.7375e-02, PNorm = 46.0423, GNorm = 0.6913, lr_0 = 6.4376e-04
Validation auc = 0.712207
Epoch 265
Validation auc = 0.715023
Epoch 266
Validation auc = 0.714554
Epoch 267
Validation auc = 0.719718
Epoch 268
Validation auc = 0.718779
Epoch 269
Loss = 6.1960e-02, PNorm = 46.0799, GNorm = 2.2363, lr_0 = 6.3826e-04
Validation auc = 0.721127
Epoch 270
Validation auc = 0.710329
Epoch 271
Validation auc = 0.713146
Epoch 272
Validation auc = 0.720188
Epoch 273
Validation auc = 0.729108
Epoch 274
Loss = 4.9970e-02, PNorm = 46.1246, GNorm = 1.8726, lr_0 = 6.3280e-04
Validation auc = 0.729577
Epoch 275
Validation auc = 0.723944
Epoch 276
Validation auc = 0.723944
Epoch 277
Validation auc = 0.708451
Epoch 278
Validation auc = 0.695305
Epoch 279
Loss = 2.7483e-02, PNorm = 46.1679, GNorm = 1.6453, lr_0 = 6.2739e-04
Validation auc = 0.689671
Epoch 280
Validation auc = 0.690610
Epoch 281
Validation auc = 0.694836
Epoch 282
Validation auc = 0.705164
Epoch 283
Validation auc = 0.707042
Epoch 284
Loss = 7.8341e-02, PNorm = 46.2081, GNorm = 1.8723, lr_0 = 6.2203e-04
Validation auc = 0.707981
Epoch 285
Validation auc = 0.711737
Epoch 286
Validation auc = 0.719249
Epoch 287
Validation auc = 0.727230
Epoch 288
Validation auc = 0.720188
Epoch 289
Loss = 2.0569e-02, PNorm = 46.2349, GNorm = 0.9366, lr_0 = 6.1671e-04
Validation auc = 0.720657
Epoch 290
Validation auc = 0.711737
Epoch 291
Validation auc = 0.701878
Epoch 292
Validation auc = 0.701408
Epoch 293
Validation auc = 0.696244
Epoch 294
Loss = 8.6827e-03, PNorm = 46.2655, GNorm = 0.2889, lr_0 = 6.1144e-04
Validation auc = 0.696244
Epoch 295
Validation auc = 0.708920
Epoch 296
Validation auc = 0.718310
Epoch 297
Validation auc = 0.712676
Epoch 298
Validation auc = 0.712207
Epoch 299
Loss = 7.5866e-03, PNorm = 46.2985, GNorm = 0.6220, lr_0 = 6.0621e-04
Validation auc = 0.715023
Model 0 best validation auc = 0.781221 on epoch 45
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.893438
Ensemble test auc = 0.893438
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.550235
Epoch 1
Validation auc = 0.520188
Epoch 2
Validation auc = 0.521127
Epoch 3
Validation auc = 0.514085
Epoch 4
Loss = 6.8373e-01, PNorm = 42.2871, GNorm = 1.2542, lr_0 = 6.5000e-04
Validation auc = 0.538967
Epoch 5
Validation auc = 0.592019
Epoch 6
Validation auc = 0.642254
Epoch 7
Validation auc = 0.641784
Epoch 8
Validation auc = 0.654930
Epoch 9
Loss = 6.2632e-01, PNorm = 42.3522, GNorm = 1.2473, lr_0 = 9.9743e-04
Validation auc = 0.636150
Epoch 10
Validation auc = 0.645540
Epoch 11
Validation auc = 0.662441
Epoch 12
Validation auc = 0.676056
Epoch 13
Validation auc = 0.724883
Epoch 14
Loss = 6.6553e-01, PNorm = 42.4524, GNorm = 2.7947, lr_0 = 9.8890e-04
Validation auc = 0.708451
Epoch 15
Validation auc = 0.701408
Epoch 16
Validation auc = 0.682160
Epoch 17
Validation auc = 0.696714
Epoch 18
Validation auc = 0.682160
Epoch 19
Loss = 6.5262e-01, PNorm = 42.5292, GNorm = 0.6861, lr_0 = 9.8045e-04
Validation auc = 0.726761
Epoch 20
Validation auc = 0.732864
Epoch 21
Validation auc = 0.726761
Epoch 22
Validation auc = 0.737089
Epoch 23
Validation auc = 0.737559
Epoch 24
Loss = 5.1586e-01, PNorm = 42.6109, GNorm = 0.6381, lr_0 = 9.7207e-04
Validation auc = 0.708920
Epoch 25
Validation auc = 0.715962
Epoch 26
Validation auc = 0.725822
Epoch 27
Validation auc = 0.748826
Epoch 28
Validation auc = 0.730986
Epoch 29
Loss = 5.3945e-01, PNorm = 42.6893, GNorm = 1.8113, lr_0 = 9.6376e-04
Validation auc = 0.755869
Epoch 30
Validation auc = 0.762441
Epoch 31
Validation auc = 0.748826
Epoch 32
Validation auc = 0.746009
Epoch 33
Validation auc = 0.767606
Epoch 34
Loss = 5.0902e-01, PNorm = 42.7618, GNorm = 1.8909, lr_0 = 9.5552e-04
Validation auc = 0.766667
Epoch 35
Validation auc = 0.750704
Epoch 36
Validation auc = 0.750235
Epoch 37
Validation auc = 0.758216
Epoch 38
Validation auc = 0.756808
Epoch 39
Loss = 4.8207e-01, PNorm = 42.8457, GNorm = 0.6001, lr_0 = 9.4735e-04
Validation auc = 0.753052
Epoch 40
Validation auc = 0.756338
Epoch 41
Validation auc = 0.761502
Epoch 42
Validation auc = 0.759624
Epoch 43
Validation auc = 0.758685
Epoch 44
Loss = 3.7865e-01, PNorm = 42.9279, GNorm = 0.6472, lr_0 = 9.3925e-04
Validation auc = 0.749765
Epoch 45
Validation auc = 0.744601
Epoch 46
Validation auc = 0.744131
Epoch 47
Validation auc = 0.738028
Epoch 48
Validation auc = 0.749296
Epoch 49
Loss = 3.8925e-01, PNorm = 43.0230, GNorm = 2.6541, lr_0 = 9.3122e-04
Validation auc = 0.740845
Epoch 50
Validation auc = 0.750235
Epoch 51
Validation auc = 0.741784
Epoch 52
Validation auc = 0.744131
Epoch 53
Validation auc = 0.738028
Epoch 54
Loss = 4.0521e-01, PNorm = 43.1115, GNorm = 3.5589, lr_0 = 9.2326e-04
Validation auc = 0.750704
Epoch 55
Validation auc = 0.756808
Epoch 56
Validation auc = 0.738967
Epoch 57
Validation auc = 0.739437
Epoch 58
Validation auc = 0.731925
Epoch 59
Loss = 3.6404e-01, PNorm = 43.2099, GNorm = 0.7273, lr_0 = 9.1537e-04
Validation auc = 0.741315
Epoch 60
Validation auc = 0.738028
Epoch 61
Validation auc = 0.729108
Epoch 62
Validation auc = 0.715023
Epoch 63
Validation auc = 0.715493
Epoch 64
Loss = 3.7938e-01, PNorm = 43.3246, GNorm = 1.8507, lr_0 = 9.0754e-04
Validation auc = 0.724883
Epoch 65
Validation auc = 0.716432
Epoch 66
Validation auc = 0.723944
Epoch 67
Validation auc = 0.710798
Epoch 68
Validation auc = 0.695775
Epoch 69
Loss = 3.2259e-01, PNorm = 43.4295, GNorm = 2.9641, lr_0 = 8.9978e-04
Validation auc = 0.698592
Epoch 70
Validation auc = 0.676526
Epoch 71
Validation auc = 0.687324
Epoch 72
Validation auc = 0.706573
Epoch 73
Validation auc = 0.696714
Epoch 74
Loss = 3.2381e-01, PNorm = 43.5311, GNorm = 3.1839, lr_0 = 8.9209e-04
Validation auc = 0.682629
Epoch 75
Validation auc = 0.707512
Epoch 76
Validation auc = 0.715023
Epoch 77
Validation auc = 0.706103
Epoch 78
Validation auc = 0.705634
Epoch 79
Loss = 2.3562e-01, PNorm = 43.6429, GNorm = 1.0891, lr_0 = 8.8447e-04
Validation auc = 0.715493
Epoch 80
Validation auc = 0.714554
Epoch 81
Validation auc = 0.704695
Epoch 82
Validation auc = 0.706573
Epoch 83
Validation auc = 0.700000
Epoch 84
Loss = 2.5425e-01, PNorm = 43.7473, GNorm = 0.6775, lr_0 = 8.7691e-04
Validation auc = 0.672770
Epoch 85
Validation auc = 0.683099
Epoch 86
Validation auc = 0.699531
Epoch 87
Validation auc = 0.687324
Epoch 88
Validation auc = 0.672300
Epoch 89
Loss = 2.6375e-01, PNorm = 43.8398, GNorm = 1.0802, lr_0 = 8.6941e-04
Validation auc = 0.680751
Epoch 90
Validation auc = 0.677465
Epoch 91
Validation auc = 0.660094
Epoch 92
Validation auc = 0.671831
Epoch 93
Validation auc = 0.685446
Epoch 94
Loss = 2.8175e-01, PNorm = 43.9322, GNorm = 1.2683, lr_0 = 8.6198e-04
Validation auc = 0.687324
Epoch 95
Validation auc = 0.699061
Epoch 96
Validation auc = 0.717371
Epoch 97
Validation auc = 0.719718
Epoch 98
Validation auc = 0.708920
Epoch 99
Loss = 1.8459e-01, PNorm = 44.0203, GNorm = 1.1945, lr_0 = 8.5461e-04
Validation auc = 0.692019
Epoch 100
Validation auc = 0.669484
Epoch 101
Validation auc = 0.665728
Epoch 102
Validation auc = 0.658685
Epoch 103
Validation auc = 0.659155
Epoch 104
Loss = 9.2797e-02, PNorm = 44.1215, GNorm = 1.0054, lr_0 = 8.4730e-04
Validation auc = 0.668075
Epoch 105
Validation auc = 0.679812
Epoch 106
Validation auc = 0.680751
Epoch 107
Validation auc = 0.705634
Epoch 108
Validation auc = 0.697653
Epoch 109
Loss = 1.8112e-01, PNorm = 44.2207, GNorm = 2.6190, lr_0 = 8.4006e-04
Validation auc = 0.696244
Epoch 110
Validation auc = 0.710329
Epoch 111
Validation auc = 0.685446
Epoch 112
Validation auc = 0.694836
Epoch 113
Validation auc = 0.701878
Epoch 114
Loss = 1.2722e-01, PNorm = 44.3036, GNorm = 0.9665, lr_0 = 8.3288e-04
Validation auc = 0.696244
Epoch 115
Validation auc = 0.700469
Epoch 116
Validation auc = 0.697653
Epoch 117
Validation auc = 0.696244
Epoch 118
Validation auc = 0.686854
Epoch 119
Loss = 1.1823e-01, PNorm = 44.3900, GNorm = 0.7320, lr_0 = 8.2576e-04
Validation auc = 0.689202
Epoch 120
Validation auc = 0.682160
Epoch 121
Validation auc = 0.693427
Epoch 122
Validation auc = 0.698122
Epoch 123
Validation auc = 0.691080
Epoch 124
Loss = 2.0422e-01, PNorm = 44.4769, GNorm = 2.5736, lr_0 = 8.1870e-04
Validation auc = 0.695775
Epoch 125
Validation auc = 0.683099
Epoch 126
Validation auc = 0.659624
Epoch 127
Validation auc = 0.656808
Epoch 128
Validation auc = 0.648357
Epoch 129
Loss = 1.8243e-01, PNorm = 44.5547, GNorm = 4.0184, lr_0 = 8.1170e-04
Validation auc = 0.654930
Epoch 130
Validation auc = 0.683099
Epoch 131
Validation auc = 0.668075
Epoch 132
Validation auc = 0.677934
Epoch 133
Validation auc = 0.696714
Epoch 134
Loss = 1.4885e-01, PNorm = 44.6264, GNorm = 2.4637, lr_0 = 8.0476e-04
Validation auc = 0.687793
Epoch 135
Validation auc = 0.679812
Epoch 136
Validation auc = 0.684038
Epoch 137
Validation auc = 0.690141
Epoch 138
Validation auc = 0.681221
Epoch 139
Loss = 1.7499e-01, PNorm = 44.6968, GNorm = 1.0863, lr_0 = 7.9788e-04
Validation auc = 0.683099
Epoch 140
Validation auc = 0.677934
Epoch 141
Validation auc = 0.684507
Epoch 142
Validation auc = 0.688732
Epoch 143
Validation auc = 0.694366
Epoch 144
Loss = 1.2722e-01, PNorm = 44.7660, GNorm = 1.9329, lr_0 = 7.9106e-04
Validation auc = 0.676995
Epoch 145
Validation auc = 0.661502
Epoch 146
Validation auc = 0.669014
Epoch 147
Validation auc = 0.669484
Epoch 148
Validation auc = 0.657277
Epoch 149
Loss = 1.2788e-01, PNorm = 44.8418, GNorm = 3.1039, lr_0 = 7.8430e-04
Validation auc = 0.660563
Epoch 150
Validation auc = 0.695305
Epoch 151
Validation auc = 0.680751
Epoch 152
Validation auc = 0.667136
Epoch 153
Validation auc = 0.673709
Epoch 154
Loss = 2.1083e-01, PNorm = 44.9106, GNorm = 3.1081, lr_0 = 7.7759e-04
Validation auc = 0.684977
Epoch 155
Validation auc = 0.667136
Epoch 156
Validation auc = 0.662911
Epoch 157
Validation auc = 0.680751
Epoch 158
Validation auc = 0.688263
Epoch 159
Loss = 1.3521e-01, PNorm = 44.9839, GNorm = 1.2381, lr_0 = 7.7095e-04
Validation auc = 0.697183
Epoch 160
Validation auc = 0.700000
Epoch 161
Validation auc = 0.713146
Epoch 162
Validation auc = 0.711268
Epoch 163
Validation auc = 0.701878
Epoch 164
Loss = 5.9492e-02, PNorm = 45.0558, GNorm = 1.6534, lr_0 = 7.6436e-04
Validation auc = 0.697183
Epoch 165
Validation auc = 0.695305
Epoch 166
Validation auc = 0.696244
Epoch 167
Validation auc = 0.693427
Epoch 168
Validation auc = 0.686385
Epoch 169
Loss = 3.5057e-02, PNorm = 45.1216, GNorm = 0.2803, lr_0 = 7.5782e-04
Validation auc = 0.681221
Epoch 170
Validation auc = 0.687324
Epoch 171
Validation auc = 0.700469
Epoch 172
Validation auc = 0.706573
Epoch 173
Validation auc = 0.695775
Epoch 174
Loss = 8.8543e-02, PNorm = 45.1799, GNorm = 2.6190, lr_0 = 7.5134e-04
Validation auc = 0.697653
Epoch 175
Validation auc = 0.706103
Epoch 176
Validation auc = 0.694836
Epoch 177
Validation auc = 0.678873
Epoch 178
Validation auc = 0.689202
Epoch 179
Loss = 7.2387e-02, PNorm = 45.2282, GNorm = 2.3791, lr_0 = 7.4492e-04
Validation auc = 0.700000
Epoch 180
Validation auc = 0.694836
Epoch 181
Validation auc = 0.684507
Epoch 182
Validation auc = 0.681690
Epoch 183
Validation auc = 0.689671
Epoch 184
Loss = 5.7125e-02, PNorm = 45.2859, GNorm = 0.6709, lr_0 = 7.3855e-04
Validation auc = 0.694836
Epoch 185
Validation auc = 0.692488
Epoch 186
Validation auc = 0.686385
Epoch 187
Validation auc = 0.686854
Epoch 188
Validation auc = 0.690610
Epoch 189
Loss = 6.3840e-02, PNorm = 45.3450, GNorm = 0.9291, lr_0 = 7.3224e-04
Validation auc = 0.700939
Epoch 190
Validation auc = 0.703756
Epoch 191
Validation auc = 0.702817
Epoch 192
Validation auc = 0.701408
Epoch 193
Validation auc = 0.704225
Epoch 194
Loss = 7.3965e-02, PNorm = 45.3939, GNorm = 3.5815, lr_0 = 7.2598e-04
Validation auc = 0.689671
Epoch 195
Validation auc = 0.670892
Epoch 196
Validation auc = 0.661502
Epoch 197
Validation auc = 0.670423
Epoch 198
Validation auc = 0.670892
Epoch 199
Loss = 9.2349e-02, PNorm = 45.4385, GNorm = 0.9681, lr_0 = 7.1977e-04
Validation auc = 0.667136
Epoch 200
Validation auc = 0.677934
Epoch 201
Validation auc = 0.697183
Epoch 202
Validation auc = 0.715493
Epoch 203
Validation auc = 0.716901
Epoch 204
Loss = 7.1835e-02, PNorm = 45.4824, GNorm = 0.4941, lr_0 = 7.1362e-04
Validation auc = 0.710798
Epoch 205
Validation auc = 0.711737
Epoch 206
Validation auc = 0.705634
Epoch 207
Validation auc = 0.706573
Epoch 208
Validation auc = 0.711268
Epoch 209
Loss = 5.6155e-02, PNorm = 45.5250, GNorm = 1.4225, lr_0 = 7.0752e-04
Validation auc = 0.704695
Epoch 210
Validation auc = 0.705164
Epoch 211
Validation auc = 0.706103
Epoch 212
Validation auc = 0.706573
Epoch 213
Validation auc = 0.702347
Epoch 214
Loss = 5.3995e-02, PNorm = 45.5711, GNorm = 1.4552, lr_0 = 7.0147e-04
Validation auc = 0.700000
Epoch 215
Validation auc = 0.699061
Epoch 216
Validation auc = 0.696714
Epoch 217
Validation auc = 0.698122
Epoch 218
Validation auc = 0.697653
Epoch 219
Loss = 3.7555e-02, PNorm = 45.6176, GNorm = 2.2384, lr_0 = 6.9548e-04
Validation auc = 0.686385
Epoch 220
Validation auc = 0.682629
Epoch 221
Validation auc = 0.692958
Epoch 222
Validation auc = 0.698592
Epoch 223
Validation auc = 0.698122
Epoch 224
Loss = 3.7107e-02, PNorm = 45.6603, GNorm = 2.2444, lr_0 = 6.8953e-04
Validation auc = 0.695305
Epoch 225
Validation auc = 0.699531
Epoch 226
Validation auc = 0.708451
Epoch 227
Validation auc = 0.709859
Epoch 228
Validation auc = 0.706573
Epoch 229
Loss = 3.6791e-02, PNorm = 45.6985, GNorm = 1.1844, lr_0 = 6.8364e-04
Validation auc = 0.699531
Epoch 230
Validation auc = 0.696244
Epoch 231
Validation auc = 0.703756
Epoch 232
Validation auc = 0.706573
Epoch 233
Validation auc = 0.703756
Epoch 234
Loss = 1.7747e-01, PNorm = 45.7425, GNorm = 2.4000, lr_0 = 6.7779e-04
Validation auc = 0.707512
Epoch 235
Validation auc = 0.717371
Epoch 236
Validation auc = 0.722066
Epoch 237
Validation auc = 0.720657
Epoch 238
Validation auc = 0.712207
Epoch 239
Loss = 1.4787e-01, PNorm = 45.7908, GNorm = 2.1591, lr_0 = 6.7200e-04
Validation auc = 0.705634
Epoch 240
Validation auc = 0.699531
Epoch 241
Validation auc = 0.697653
Epoch 242
Validation auc = 0.694836
Epoch 243
Validation auc = 0.684977
Epoch 244
Loss = 4.8396e-02, PNorm = 45.8234, GNorm = 1.1231, lr_0 = 6.6625e-04
Validation auc = 0.684038
Epoch 245
Validation auc = 0.682629
Epoch 246
Validation auc = 0.687324
Epoch 247
Validation auc = 0.692958
Epoch 248
Validation auc = 0.697183
Epoch 249
Loss = 1.8907e-02, PNorm = 45.8545, GNorm = 0.9500, lr_0 = 6.6056e-04
Validation auc = 0.699061
Epoch 250
Validation auc = 0.701878
Epoch 251
Validation auc = 0.700469
Epoch 252
Validation auc = 0.696714
Epoch 253
Validation auc = 0.690610
Epoch 254
Loss = 5.0833e-02, PNorm = 45.8865, GNorm = 0.5387, lr_0 = 6.5491e-04
Validation auc = 0.684977
Epoch 255
Validation auc = 0.684977
Epoch 256
Validation auc = 0.689671
Epoch 257
Validation auc = 0.692488
Epoch 258
Validation auc = 0.690610
Epoch 259
Loss = 2.9464e-02, PNorm = 45.9162, GNorm = 0.6525, lr_0 = 6.4931e-04
Validation auc = 0.685915
Epoch 260
Validation auc = 0.684977
Epoch 261
Validation auc = 0.693427
Epoch 262
Validation auc = 0.700939
Epoch 263
Validation auc = 0.705634
Epoch 264
Loss = 3.0654e-02, PNorm = 45.9540, GNorm = 0.1844, lr_0 = 6.4376e-04
Validation auc = 0.698122
Epoch 265
Validation auc = 0.695775
Epoch 266
Validation auc = 0.698592
Epoch 267
Validation auc = 0.703286
Epoch 268
Validation auc = 0.701878
Epoch 269
Loss = 3.9360e-02, PNorm = 45.9885, GNorm = 1.2055, lr_0 = 6.3826e-04
Validation auc = 0.695305
Epoch 270
Validation auc = 0.680751
Epoch 271
Validation auc = 0.672770
Epoch 272
Validation auc = 0.684507
Epoch 273
Validation auc = 0.685446
Epoch 274
Loss = 6.6573e-02, PNorm = 46.0229, GNorm = 0.8298, lr_0 = 6.3280e-04
Validation auc = 0.685446
Epoch 275
Validation auc = 0.689671
Epoch 276
Validation auc = 0.687324
Epoch 277
Validation auc = 0.688263
Epoch 278
Validation auc = 0.694366
Epoch 279
Loss = 1.5894e-02, PNorm = 46.0598, GNorm = 0.3328, lr_0 = 6.2739e-04
Validation auc = 0.701878
Epoch 280
Validation auc = 0.697653
Epoch 281
Validation auc = 0.692488
Epoch 282
Validation auc = 0.687793
Epoch 283
Validation auc = 0.694366
Epoch 284
Loss = 2.9619e-02, PNorm = 46.0898, GNorm = 0.4589, lr_0 = 6.2203e-04
Validation auc = 0.698122
Epoch 285
Validation auc = 0.691549
Epoch 286
Validation auc = 0.685915
Epoch 287
Validation auc = 0.678873
Epoch 288
Validation auc = 0.682160
Epoch 289
Loss = 9.7657e-02, PNorm = 46.1222, GNorm = 1.2441, lr_0 = 6.1671e-04
Validation auc = 0.704695
Epoch 290
Validation auc = 0.711737
Epoch 291
Validation auc = 0.692488
Epoch 292
Validation auc = 0.675117
Epoch 293
Validation auc = 0.675587
Epoch 294
Loss = 5.3315e-02, PNorm = 46.1695, GNorm = 0.6685, lr_0 = 6.1144e-04
Validation auc = 0.681221
Epoch 295
Validation auc = 0.686385
Epoch 296
Validation auc = 0.689671
Epoch 297
Validation auc = 0.687793
Epoch 298
Validation auc = 0.689202
Epoch 299
Loss = 2.7503e-02, PNorm = 46.2168, GNorm = 0.5722, lr_0 = 6.0621e-04
Validation auc = 0.697183
Model 0 best validation auc = 0.767606 on epoch 33
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.886815
Ensemble test auc = 0.886815
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.559624
Epoch 1
Validation auc = 0.566667
Epoch 2
Validation auc = 0.550704
Epoch 3
Validation auc = 0.605634
Epoch 4
Loss = 6.9646e-01, PNorm = 42.2866, GNorm = 1.2347, lr_0 = 6.5000e-04
Validation auc = 0.638498
Epoch 5
Validation auc = 0.646479
Epoch 6
Validation auc = 0.620657
Epoch 7
Validation auc = 0.626291
Epoch 8
Validation auc = 0.589202
Epoch 9
Loss = 6.3140e-01, PNorm = 42.3541, GNorm = 0.8560, lr_0 = 9.9743e-04
Validation auc = 0.637089
Epoch 10
Validation auc = 0.633333
Epoch 11
Validation auc = 0.638967
Epoch 12
Validation auc = 0.686385
Epoch 13
Validation auc = 0.706573
Epoch 14
Loss = 5.7404e-01, PNorm = 42.4603, GNorm = 0.8584, lr_0 = 9.8890e-04
Validation auc = 0.709859
Epoch 15
Validation auc = 0.730047
Epoch 16
Validation auc = 0.712207
Epoch 17
Validation auc = 0.735211
Epoch 18
Validation auc = 0.721127
Epoch 19
Loss = 5.5913e-01, PNorm = 42.5388, GNorm = 1.3456, lr_0 = 9.8045e-04
Validation auc = 0.712207
Epoch 20
Validation auc = 0.731455
Epoch 21
Validation auc = 0.729108
Epoch 22
Validation auc = 0.728169
Epoch 23
Validation auc = 0.753521
Epoch 24
Loss = 5.1418e-01, PNorm = 42.6164, GNorm = 1.6396, lr_0 = 9.7207e-04
Validation auc = 0.747887
Epoch 25
Validation auc = 0.739906
Epoch 26
Validation auc = 0.751174
Epoch 27
Validation auc = 0.743662
Epoch 28
Validation auc = 0.746009
Epoch 29
Loss = 4.6885e-01, PNorm = 42.6956, GNorm = 1.2037, lr_0 = 9.6376e-04
Validation auc = 0.729577
Epoch 30
Validation auc = 0.728169
Epoch 31
Validation auc = 0.744601
Epoch 32
Validation auc = 0.731455
Epoch 33
Validation auc = 0.736150
Epoch 34
Loss = 4.7196e-01, PNorm = 42.7754, GNorm = 0.7142, lr_0 = 9.5552e-04
Validation auc = 0.748826
Epoch 35
Validation auc = 0.752582
Epoch 36
Validation auc = 0.755869
Epoch 37
Validation auc = 0.769953
Epoch 38
Validation auc = 0.747418
Epoch 39
Loss = 3.8944e-01, PNorm = 42.8597, GNorm = 0.7825, lr_0 = 9.4735e-04
Validation auc = 0.763850
Epoch 40
Validation auc = 0.750235
Epoch 41
Validation auc = 0.748826
Epoch 42
Validation auc = 0.758216
Epoch 43
Validation auc = 0.737089
Epoch 44
Loss = 4.1049e-01, PNorm = 42.9483, GNorm = 1.8279, lr_0 = 9.3925e-04
Validation auc = 0.749296
Epoch 45
Validation auc = 0.738967
Epoch 46
Validation auc = 0.747887
Epoch 47
Validation auc = 0.743192
Epoch 48
Validation auc = 0.740845
Epoch 49
Loss = 3.4580e-01, PNorm = 43.0332, GNorm = 0.8210, lr_0 = 9.3122e-04
Validation auc = 0.746948
Epoch 50
Validation auc = 0.739437
Epoch 51
Validation auc = 0.742723
Epoch 52
Validation auc = 0.713146
Epoch 53
Validation auc = 0.743662
Epoch 54
Loss = 3.8246e-01, PNorm = 43.1145, GNorm = 0.7822, lr_0 = 9.2326e-04
Validation auc = 0.730047
Epoch 55
Validation auc = 0.735211
Epoch 56
Validation auc = 0.714554
Epoch 57
Validation auc = 0.704225
Epoch 58
Validation auc = 0.695775
Epoch 59
Loss = 4.2640e-01, PNorm = 43.2018, GNorm = 1.7529, lr_0 = 9.1537e-04
Validation auc = 0.725822
Epoch 60
Validation auc = 0.710798
Epoch 61
Validation auc = 0.727230
Epoch 62
Validation auc = 0.706103
Epoch 63
Validation auc = 0.706103
Epoch 64
Loss = 2.9648e-01, PNorm = 43.2981, GNorm = 2.7809, lr_0 = 9.0754e-04
Validation auc = 0.690610
Epoch 65
Validation auc = 0.710798
Epoch 66
Validation auc = 0.696244
Epoch 67
Validation auc = 0.700939
Epoch 68
Validation auc = 0.704695
Epoch 69
Loss = 3.6812e-01, PNorm = 43.3957, GNorm = 4.2694, lr_0 = 8.9978e-04
Validation auc = 0.687793
Epoch 70
Validation auc = 0.708451
Epoch 71
Validation auc = 0.699061
Epoch 72
Validation auc = 0.707981
Epoch 73
Validation auc = 0.700469
Epoch 74
Loss = 2.7065e-01, PNorm = 43.5032, GNorm = 1.3986, lr_0 = 8.9209e-04
Validation auc = 0.685915
Epoch 75
Validation auc = 0.714085
Epoch 76
Validation auc = 0.692958
Epoch 77
Validation auc = 0.709390
Epoch 78
Validation auc = 0.728638
Epoch 79
Loss = 4.2993e-01, PNorm = 43.6088, GNorm = 4.0159, lr_0 = 8.8447e-04
Validation auc = 0.699531
Epoch 80
Validation auc = 0.663850
Epoch 81
Validation auc = 0.670892
Epoch 82
Validation auc = 0.678404
Epoch 83
Validation auc = 0.662441
Epoch 84
Loss = 3.7269e-01, PNorm = 43.7122, GNorm = 2.5778, lr_0 = 8.7691e-04
Validation auc = 0.664319
Epoch 85
Validation auc = 0.670892
Epoch 86
Validation auc = 0.686854
Epoch 87
Validation auc = 0.689671
Epoch 88
Validation auc = 0.682629
Epoch 89
Loss = 2.4035e-01, PNorm = 43.8293, GNorm = 1.1682, lr_0 = 8.6941e-04
Validation auc = 0.682629
Epoch 90
Validation auc = 0.689671
Epoch 91
Validation auc = 0.698592
Epoch 92
Validation auc = 0.681221
Epoch 93
Validation auc = 0.653521
Epoch 94
Loss = 3.1391e-01, PNorm = 43.9435, GNorm = 5.7289, lr_0 = 8.6198e-04
Validation auc = 0.679812
Epoch 95
Validation auc = 0.696244
Epoch 96
Validation auc = 0.667136
Epoch 97
Validation auc = 0.678404
Epoch 98
Validation auc = 0.694366
Epoch 99
Loss = 3.1652e-01, PNorm = 44.0447, GNorm = 1.6379, lr_0 = 8.5461e-04
Validation auc = 0.659155
Epoch 100
Validation auc = 0.648826
Epoch 101
Validation auc = 0.654460
Epoch 102
Validation auc = 0.668545
Epoch 103
Validation auc = 0.668545
Epoch 104
Loss = 1.8959e-01, PNorm = 44.1493, GNorm = 0.5549, lr_0 = 8.4730e-04
Validation auc = 0.659624
Epoch 105
Validation auc = 0.668545
Epoch 106
Validation auc = 0.689202
Epoch 107
Validation auc = 0.691080
Epoch 108
Validation auc = 0.680282
Epoch 109
Loss = 1.3488e-01, PNorm = 44.2458, GNorm = 1.5994, lr_0 = 8.4006e-04
Validation auc = 0.673709
Epoch 110
Validation auc = 0.687324
Epoch 111
Validation auc = 0.686854
Epoch 112
Validation auc = 0.683568
Epoch 113
Validation auc = 0.678404
Epoch 114
Loss = 1.7919e-01, PNorm = 44.3394, GNorm = 2.6544, lr_0 = 8.3288e-04
Validation auc = 0.684507
Epoch 115
Validation auc = 0.693427
Epoch 116
Validation auc = 0.683099
Epoch 117
Validation auc = 0.678873
Epoch 118
Validation auc = 0.688263
Epoch 119
Loss = 1.7480e-01, PNorm = 44.4245, GNorm = 4.5749, lr_0 = 8.2576e-04
Validation auc = 0.684507
Epoch 120
Validation auc = 0.667606
Epoch 121
Validation auc = 0.682160
Epoch 122
Validation auc = 0.685915
Epoch 123
Validation auc = 0.681221
Epoch 124
Loss = 2.1459e-01, PNorm = 44.5024, GNorm = 1.4406, lr_0 = 8.1870e-04
Validation auc = 0.686385
Epoch 125
Validation auc = 0.694836
Epoch 126
Validation auc = 0.691549
Epoch 127
Validation auc = 0.687324
Epoch 128
Validation auc = 0.691549
Epoch 129
Loss = 1.4393e-01, PNorm = 44.5905, GNorm = 1.9284, lr_0 = 8.1170e-04
Validation auc = 0.687324
Epoch 130
Validation auc = 0.691080
Epoch 131
Validation auc = 0.695775
Epoch 132
Validation auc = 0.693427
Epoch 133
Validation auc = 0.696714
Epoch 134
Loss = 1.6619e-01, PNorm = 44.6812, GNorm = 1.3566, lr_0 = 8.0476e-04
Validation auc = 0.707042
Epoch 135
Validation auc = 0.689202
Epoch 136
Validation auc = 0.663380
Epoch 137
Validation auc = 0.651174
Epoch 138
Validation auc = 0.651643
Epoch 139
Loss = 2.0889e-01, PNorm = 44.7623, GNorm = 3.3104, lr_0 = 7.9788e-04
Validation auc = 0.665728
Epoch 140
Validation auc = 0.664789
Epoch 141
Validation auc = 0.679812
Epoch 142
Validation auc = 0.699531
Epoch 143
Validation auc = 0.693427
Epoch 144
Loss = 8.8759e-02, PNorm = 44.8346, GNorm = 0.9839, lr_0 = 7.9106e-04
Validation auc = 0.686385
Epoch 145
Validation auc = 0.689202
Epoch 146
Validation auc = 0.696244
Epoch 147
Validation auc = 0.673239
Epoch 148
Validation auc = 0.656338
Epoch 149
Loss = 1.4371e-01, PNorm = 44.8965, GNorm = 1.2381, lr_0 = 7.8430e-04
Validation auc = 0.659155
Epoch 150
Validation auc = 0.661972
Epoch 151
Validation auc = 0.651174
Epoch 152
Validation auc = 0.647418
Epoch 153
Validation auc = 0.658216
Epoch 154
Loss = 9.8748e-02, PNorm = 44.9686, GNorm = 2.0986, lr_0 = 7.7759e-04
Validation auc = 0.671362
Epoch 155
Validation auc = 0.673709
Epoch 156
Validation auc = 0.671362
Epoch 157
Validation auc = 0.669484
Epoch 158
Validation auc = 0.670892
Epoch 159
Loss = 1.5234e-01, PNorm = 45.0457, GNorm = 3.7009, lr_0 = 7.7095e-04
Validation auc = 0.670423
Epoch 160
Validation auc = 0.652113
Epoch 161
Validation auc = 0.649765
Epoch 162
Validation auc = 0.649296
Epoch 163
Validation auc = 0.632864
Epoch 164
Loss = 7.0053e-02, PNorm = 45.1271, GNorm = 2.4902, lr_0 = 7.6436e-04
Validation auc = 0.633333
Epoch 165
Validation auc = 0.647418
Epoch 166
Validation auc = 0.649296
Epoch 167
Validation auc = 0.653521
Epoch 168
Validation auc = 0.684977
Epoch 169
Loss = 1.7944e-01, PNorm = 45.1978, GNorm = 7.3052, lr_0 = 7.5782e-04
Validation auc = 0.695775
Epoch 170
Validation auc = 0.697183
Epoch 171
Validation auc = 0.706103
Epoch 172
Validation auc = 0.707981
Epoch 173
Validation auc = 0.703756
Epoch 174
Loss = 1.1273e-01, PNorm = 45.2690, GNorm = 2.0269, lr_0 = 7.5134e-04
Validation auc = 0.698122
Epoch 175
Validation auc = 0.701408
Epoch 176
Validation auc = 0.703756
Epoch 177
Validation auc = 0.677465
Epoch 178
Validation auc = 0.667606
Epoch 179
Loss = 4.2289e-02, PNorm = 45.3366, GNorm = 0.7962, lr_0 = 7.4492e-04
Validation auc = 0.666197
Epoch 180
Validation auc = 0.665728
Epoch 181
Validation auc = 0.658685
Epoch 182
Validation auc = 0.654460
Epoch 183
Validation auc = 0.654930
Epoch 184
Loss = 5.4719e-02, PNorm = 45.3941, GNorm = 1.6588, lr_0 = 7.3855e-04
Validation auc = 0.669953
Epoch 185
Validation auc = 0.679343
Epoch 186
Validation auc = 0.679812
Epoch 187
Validation auc = 0.679812
Epoch 188
Validation auc = 0.683568
Epoch 189
Loss = 2.0627e-02, PNorm = 45.4532, GNorm = 0.5899, lr_0 = 7.3224e-04
Validation auc = 0.686385
Epoch 190
Validation auc = 0.692958
Epoch 191
Validation auc = 0.687324
Epoch 192
Validation auc = 0.692488
Epoch 193
Validation auc = 0.694366
Epoch 194
Loss = 4.1262e-02, PNorm = 45.5136, GNorm = 1.6573, lr_0 = 7.2598e-04
Validation auc = 0.699531
Epoch 195
Validation auc = 0.694836
Epoch 196
Validation auc = 0.695775
Epoch 197
Validation auc = 0.707042
Epoch 198
Validation auc = 0.706573
Epoch 199
Loss = 8.7862e-02, PNorm = 45.5672, GNorm = 1.4983, lr_0 = 7.1977e-04
Validation auc = 0.701408
Epoch 200
Validation auc = 0.697653
Epoch 201
Validation auc = 0.692488
Epoch 202
Validation auc = 0.688732
Epoch 203
Validation auc = 0.682160
Epoch 204
Loss = 2.5200e-02, PNorm = 45.6183, GNorm = 0.4316, lr_0 = 7.1362e-04
Validation auc = 0.669953
Epoch 205
Validation auc = 0.669014
Epoch 206
Validation auc = 0.668545
Epoch 207
Validation auc = 0.667136
Epoch 208
Validation auc = 0.667136
Epoch 209
Loss = 1.1414e-01, PNorm = 45.6707, GNorm = 2.8694, lr_0 = 7.0752e-04
Validation auc = 0.667606
Epoch 210
Validation auc = 0.661502
Epoch 211
Validation auc = 0.669014
Epoch 212
Validation auc = 0.669484
Epoch 213
Validation auc = 0.668075
Epoch 214
Loss = 8.8275e-02, PNorm = 45.7167, GNorm = 1.4608, lr_0 = 7.0147e-04
Validation auc = 0.665258
Epoch 215
Validation auc = 0.673239
Epoch 216
Validation auc = 0.673709
Epoch 217
Validation auc = 0.678873
Epoch 218
Validation auc = 0.683568
Epoch 219
Loss = 5.3362e-02, PNorm = 45.7631, GNorm = 0.2178, lr_0 = 6.9548e-04
Validation auc = 0.681690
Epoch 220
Validation auc = 0.690141
Epoch 221
Validation auc = 0.695305
Epoch 222
Validation auc = 0.694836
Epoch 223
Validation auc = 0.697653
Epoch 224
Loss = 6.6566e-02, PNorm = 45.8096, GNorm = 2.0486, lr_0 = 6.8953e-04
Validation auc = 0.706103
Epoch 225
Validation auc = 0.718779
Epoch 226
Validation auc = 0.703756
Epoch 227
Validation auc = 0.695775
Epoch 228
Validation auc = 0.692019
Epoch 229
Loss = 5.3641e-02, PNorm = 45.8488, GNorm = 1.6623, lr_0 = 6.8364e-04
Validation auc = 0.694366
Epoch 230
Validation auc = 0.688263
Epoch 231
Validation auc = 0.684507
Epoch 232
Validation auc = 0.683568
Epoch 233
Validation auc = 0.692958
Epoch 234
Loss = 2.0386e-02, PNorm = 45.8903, GNorm = 0.8055, lr_0 = 6.7779e-04
Validation auc = 0.697183
Epoch 235
Validation auc = 0.690141
Epoch 236
Validation auc = 0.676995
Epoch 237
Validation auc = 0.688263
Epoch 238
Validation auc = 0.695775
Epoch 239
Loss = 8.2198e-02, PNorm = 45.9381, GNorm = 2.4197, lr_0 = 6.7200e-04
Validation auc = 0.684977
Epoch 240
Validation auc = 0.680751
Epoch 241
Validation auc = 0.698592
Epoch 242
Validation auc = 0.712676
Epoch 243
Validation auc = 0.719249
Epoch 244
Loss = 8.8593e-02, PNorm = 45.9911, GNorm = 1.0563, lr_0 = 6.6625e-04
Validation auc = 0.716901
Epoch 245
Validation auc = 0.720657
Epoch 246
Validation auc = 0.723005
Epoch 247
Validation auc = 0.725822
Epoch 248
Validation auc = 0.719718
Epoch 249
Loss = 3.8272e-02, PNorm = 46.0395, GNorm = 0.8598, lr_0 = 6.6056e-04
Validation auc = 0.707981
Epoch 250
Validation auc = 0.699531
Epoch 251
Validation auc = 0.696714
Epoch 252
Validation auc = 0.701878
Epoch 253
Validation auc = 0.701408
Epoch 254
Loss = 2.9702e-02, PNorm = 46.0858, GNorm = 0.3593, lr_0 = 6.5491e-04
Validation auc = 0.699531
Epoch 255
Validation auc = 0.694366
Epoch 256
Validation auc = 0.701878
Epoch 257
Validation auc = 0.711737
Epoch 258
Validation auc = 0.703756
Epoch 259
Loss = 1.7803e-02, PNorm = 46.1283, GNorm = 0.5890, lr_0 = 6.4931e-04
Validation auc = 0.692019
Epoch 260
Validation auc = 0.686854
Epoch 261
Validation auc = 0.684977
Epoch 262
Validation auc = 0.686854
Epoch 263
Validation auc = 0.682629
Epoch 264
Loss = 5.9190e-02, PNorm = 46.1655, GNorm = 1.6297, lr_0 = 6.4376e-04
Validation auc = 0.684507
Epoch 265
Validation auc = 0.690141
Epoch 266
Validation auc = 0.701878
Epoch 267
Validation auc = 0.710798
Epoch 268
Validation auc = 0.700000
Epoch 269
Loss = 8.2308e-02, PNorm = 46.2006, GNorm = 1.2062, lr_0 = 6.3826e-04
Validation auc = 0.692488
Epoch 270
Validation auc = 0.692488
Epoch 271
Validation auc = 0.697653
Epoch 272
Validation auc = 0.689671
Epoch 273
Validation auc = 0.678873
Epoch 274
Loss = 2.9659e-02, PNorm = 46.2426, GNorm = 0.8406, lr_0 = 6.3280e-04
Validation auc = 0.677465
Epoch 275
Validation auc = 0.681221
Epoch 276
Validation auc = 0.685915
Epoch 277
Validation auc = 0.686385
Epoch 278
Validation auc = 0.670892
Epoch 279
Loss = 4.5448e-02, PNorm = 46.2824, GNorm = 2.4125, lr_0 = 6.2739e-04
Validation auc = 0.676526
Epoch 280
Validation auc = 0.687324
Epoch 281
Validation auc = 0.706573
Epoch 282
Validation auc = 0.707981
Epoch 283
Validation auc = 0.697653
Epoch 284
Loss = 7.1381e-02, PNorm = 46.3273, GNorm = 2.5229, lr_0 = 6.2203e-04
Validation auc = 0.694836
Epoch 285
Validation auc = 0.704225
Epoch 286
Validation auc = 0.707042
Epoch 287
Validation auc = 0.716901
Epoch 288
Validation auc = 0.714554
Epoch 289
Loss = 1.0879e-02, PNorm = 46.3767, GNorm = 0.4145, lr_0 = 6.1671e-04
Validation auc = 0.703756
Epoch 290
Validation auc = 0.704695
Epoch 291
Validation auc = 0.704695
Epoch 292
Validation auc = 0.707042
Epoch 293
Validation auc = 0.715023
Epoch 294
Loss = 4.2610e-02, PNorm = 46.4167, GNorm = 2.7536, lr_0 = 6.1144e-04
Validation auc = 0.713146
Epoch 295
Validation auc = 0.700939
Epoch 296
Validation auc = 0.689671
Epoch 297
Validation auc = 0.693427
Epoch 298
Validation auc = 0.702347
Epoch 299
Loss = 2.2985e-02, PNorm = 46.4627, GNorm = 0.7609, lr_0 = 6.0621e-04
Validation auc = 0.713615
Model 0 best validation auc = 0.769953 on epoch 37
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.863937
Ensemble test auc = 0.863937
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.659624
Epoch 1
Validation auc = 0.730516
Epoch 2
Validation auc = 0.737089
Epoch 3
Validation auc = 0.651174
Epoch 4
Loss = 6.8133e-01, PNorm = 42.2870, GNorm = 1.3379, lr_0 = 6.5000e-04
Validation auc = 0.683568
Epoch 5
Validation auc = 0.661972
Epoch 6
Validation auc = 0.623944
Epoch 7
Validation auc = 0.645070
Epoch 8
Validation auc = 0.637559
Epoch 9
Loss = 6.6639e-01, PNorm = 42.3556, GNorm = 1.4464, lr_0 = 9.9743e-04
Validation auc = 0.603756
Epoch 10
Validation auc = 0.607512
Epoch 11
Validation auc = 0.667606
Epoch 12
Validation auc = 0.700469
Epoch 13
Validation auc = 0.685915
Epoch 14
Loss = 5.6895e-01, PNorm = 42.4581, GNorm = 1.1201, lr_0 = 9.8890e-04
Validation auc = 0.707981
Epoch 15
Validation auc = 0.693897
Epoch 16
Validation auc = 0.723474
Epoch 17
Validation auc = 0.690141
Epoch 18
Validation auc = 0.736620
Epoch 19
Loss = 5.5884e-01, PNorm = 42.5428, GNorm = 0.8311, lr_0 = 9.8045e-04
Validation auc = 0.726291
Epoch 20
Validation auc = 0.755399
Epoch 21
Validation auc = 0.768075
Epoch 22
Validation auc = 0.727700
Epoch 23
Validation auc = 0.752113
Epoch 24
Loss = 5.2655e-01, PNorm = 42.6212, GNorm = 1.0207, lr_0 = 9.7207e-04
Validation auc = 0.763850
Epoch 25
Validation auc = 0.728169
Epoch 26
Validation auc = 0.758685
Epoch 27
Validation auc = 0.768545
Epoch 28
Validation auc = 0.744131
Epoch 29
Loss = 5.1881e-01, PNorm = 42.6946, GNorm = 2.9760, lr_0 = 9.6376e-04
Validation auc = 0.742723
Epoch 30
Validation auc = 0.769484
Epoch 31
Validation auc = 0.763850
Epoch 32
Validation auc = 0.753991
Epoch 33
Validation auc = 0.764789
Epoch 34
Loss = 4.5695e-01, PNorm = 42.7733, GNorm = 2.5917, lr_0 = 9.5552e-04
Validation auc = 0.749765
Epoch 35
Validation auc = 0.762911
Epoch 36
Validation auc = 0.770892
Epoch 37
Validation auc = 0.772770
Epoch 38
Validation auc = 0.780282
Epoch 39
Loss = 4.7218e-01, PNorm = 42.8367, GNorm = 0.8950, lr_0 = 9.4735e-04
Validation auc = 0.785446
Epoch 40
Validation auc = 0.781690
Epoch 41
Validation auc = 0.787793
Epoch 42
Validation auc = 0.777465
Epoch 43
Validation auc = 0.780282
Epoch 44
Loss = 4.4741e-01, PNorm = 42.9175, GNorm = 0.7007, lr_0 = 9.3925e-04
Validation auc = 0.770423
Epoch 45
Validation auc = 0.765728
Epoch 46
Validation auc = 0.768545
Epoch 47
Validation auc = 0.740376
Epoch 48
Validation auc = 0.770423
Epoch 49
Loss = 3.9688e-01, PNorm = 43.0059, GNorm = 0.8426, lr_0 = 9.3122e-04
Validation auc = 0.753521
Epoch 50
Validation auc = 0.759155
Epoch 51
Validation auc = 0.764319
Epoch 52
Validation auc = 0.722066
Epoch 53
Validation auc = 0.746948
Epoch 54
Loss = 3.9043e-01, PNorm = 43.0883, GNorm = 3.6297, lr_0 = 9.2326e-04
Validation auc = 0.776995
Epoch 55
Validation auc = 0.762911
Epoch 56
Validation auc = 0.753052
Epoch 57
Validation auc = 0.767606
Epoch 58
Validation auc = 0.776995
Epoch 59
Loss = 3.9370e-01, PNorm = 43.1843, GNorm = 1.1222, lr_0 = 9.1537e-04
Validation auc = 0.779812
Epoch 60
Validation auc = 0.768075
Epoch 61
Validation auc = 0.766667
Epoch 62
Validation auc = 0.773709
Epoch 63
Validation auc = 0.773239
Epoch 64
Loss = 3.0326e-01, PNorm = 43.2853, GNorm = 1.6779, lr_0 = 9.0754e-04
Validation auc = 0.761972
Epoch 65
Validation auc = 0.769484
Epoch 66
Validation auc = 0.772300
Epoch 67
Validation auc = 0.759624
Epoch 68
Validation auc = 0.752113
Epoch 69
Loss = 3.5440e-01, PNorm = 43.3795, GNorm = 1.7846, lr_0 = 8.9978e-04
Validation auc = 0.762911
Epoch 70
Validation auc = 0.743192
Epoch 71
Validation auc = 0.740376
Epoch 72
Validation auc = 0.743192
Epoch 73
Validation auc = 0.722535
Epoch 74
Loss = 2.4649e-01, PNorm = 43.4788, GNorm = 2.7024, lr_0 = 8.9209e-04
Validation auc = 0.711737
Epoch 75
Validation auc = 0.717840
Epoch 76
Validation auc = 0.710798
Epoch 77
Validation auc = 0.718779
Epoch 78
Validation auc = 0.732394
Epoch 79
Loss = 2.8106e-01, PNorm = 43.5751, GNorm = 1.2032, lr_0 = 8.8447e-04
Validation auc = 0.729108
Epoch 80
Validation auc = 0.719249
Epoch 81
Validation auc = 0.726761
Epoch 82
Validation auc = 0.722066
Epoch 83
Validation auc = 0.723005
Epoch 84
Loss = 2.2199e-01, PNorm = 43.6822, GNorm = 1.7263, lr_0 = 8.7691e-04
Validation auc = 0.723474
Epoch 85
Validation auc = 0.730047
Epoch 86
Validation auc = 0.722535
Epoch 87
Validation auc = 0.720188
Epoch 88
Validation auc = 0.722535
Epoch 89
Loss = 2.1175e-01, PNorm = 43.7864, GNorm = 2.3680, lr_0 = 8.6941e-04
Validation auc = 0.698122
Epoch 90
Validation auc = 0.697653
Epoch 91
Validation auc = 0.717371
Epoch 92
Validation auc = 0.702817
Epoch 93
Validation auc = 0.690610
Epoch 94
Loss = 3.0680e-01, PNorm = 43.8893, GNorm = 1.7835, lr_0 = 8.6198e-04
Validation auc = 0.698122
Epoch 95
Validation auc = 0.689202
Epoch 96
Validation auc = 0.691549
Epoch 97
Validation auc = 0.712676
Epoch 98
Validation auc = 0.729577
Epoch 99
Loss = 2.3087e-01, PNorm = 43.9995, GNorm = 2.0825, lr_0 = 8.5461e-04
Validation auc = 0.731455
Epoch 100
Validation auc = 0.729108
Epoch 101
Validation auc = 0.733803
Epoch 102
Validation auc = 0.732394
Epoch 103
Validation auc = 0.719249
Epoch 104
Loss = 1.5768e-01, PNorm = 44.1068, GNorm = 1.7271, lr_0 = 8.4730e-04
Validation auc = 0.707042
Epoch 105
Validation auc = 0.705164
Epoch 106
Validation auc = 0.701408
Epoch 107
Validation auc = 0.701878
Epoch 108
Validation auc = 0.713615
Epoch 109
Loss = 1.3985e-01, PNorm = 44.2206, GNorm = 1.5967, lr_0 = 8.4006e-04
Validation auc = 0.704695
Epoch 110
Validation auc = 0.704695
Epoch 111
Validation auc = 0.702347
Epoch 112
Validation auc = 0.694836
Epoch 113
Validation auc = 0.698592
Epoch 114
Loss = 3.2332e-01, PNorm = 44.3172, GNorm = 1.3284, lr_0 = 8.3288e-04
Validation auc = 0.713615
Epoch 115
Validation auc = 0.686854
Epoch 116
Validation auc = 0.681221
Epoch 117
Validation auc = 0.688732
Epoch 118
Validation auc = 0.702817
Epoch 119
Loss = 1.9296e-01, PNorm = 44.4034, GNorm = 0.9774, lr_0 = 8.2576e-04
Validation auc = 0.711737
Epoch 120
Validation auc = 0.721596
Epoch 121
Validation auc = 0.734742
Epoch 122
Validation auc = 0.736150
Epoch 123
Validation auc = 0.730516
Epoch 124
Loss = 1.4625e-01, PNorm = 44.4973, GNorm = 0.7533, lr_0 = 8.1870e-04
Validation auc = 0.721596
Epoch 125
Validation auc = 0.713615
Epoch 126
Validation auc = 0.713615
Epoch 127
Validation auc = 0.718779
Epoch 128
Validation auc = 0.707981
Epoch 129
Loss = 5.8069e-02, PNorm = 44.5927, GNorm = 1.3050, lr_0 = 8.1170e-04
Validation auc = 0.696714
Epoch 130
Validation auc = 0.700469
Epoch 131
Validation auc = 0.702817
Epoch 132
Validation auc = 0.696714
Epoch 133
Validation auc = 0.690610
Epoch 134
Loss = 9.9182e-02, PNorm = 44.6746, GNorm = 1.1062, lr_0 = 8.0476e-04
Validation auc = 0.696714
Epoch 135
Validation auc = 0.718779
Epoch 136
Validation auc = 0.719249
Epoch 137
Validation auc = 0.712676
Epoch 138
Validation auc = 0.713146
Epoch 139
Loss = 6.0786e-02, PNorm = 44.7501, GNorm = 0.6431, lr_0 = 7.9788e-04
Validation auc = 0.717840
Epoch 140
Validation auc = 0.709390
Epoch 141
Validation auc = 0.707512
Epoch 142
Validation auc = 0.712207
Epoch 143
Validation auc = 0.720657
Epoch 144
Loss = 1.1532e-01, PNorm = 44.8296, GNorm = 3.2579, lr_0 = 7.9106e-04
Validation auc = 0.713146
Epoch 145
Validation auc = 0.704225
Epoch 146
Validation auc = 0.704695
Epoch 147
Validation auc = 0.720657
Epoch 148
Validation auc = 0.715493
Epoch 149
Loss = 1.2873e-01, PNorm = 44.9136, GNorm = 0.5751, lr_0 = 7.8430e-04
Validation auc = 0.700939
Epoch 150
Validation auc = 0.695775
Epoch 151
Validation auc = 0.695305
Epoch 152
Validation auc = 0.699061
Epoch 153
Validation auc = 0.702347
Epoch 154
Loss = 1.0199e-01, PNorm = 44.9917, GNorm = 2.2388, lr_0 = 7.7759e-04
Validation auc = 0.705164
Epoch 155
Validation auc = 0.720188
Epoch 156
Validation auc = 0.727230
Epoch 157
Validation auc = 0.725822
Epoch 158
Validation auc = 0.708920
Epoch 159
Loss = 1.0705e-01, PNorm = 45.0816, GNorm = 1.2026, lr_0 = 7.7095e-04
Validation auc = 0.694836
Epoch 160
Validation auc = 0.683568
Epoch 161
Validation auc = 0.674648
Epoch 162
Validation auc = 0.669953
Epoch 163
Validation auc = 0.667606
Epoch 164
Loss = 1.5284e-01, PNorm = 45.1549, GNorm = 1.9493, lr_0 = 7.6436e-04
Validation auc = 0.670423
Epoch 165
Validation auc = 0.681690
Epoch 166
Validation auc = 0.684977
Epoch 167
Validation auc = 0.694366
Epoch 168
Validation auc = 0.700939
Epoch 169
Loss = 1.0744e-01, PNorm = 45.2283, GNorm = 1.6381, lr_0 = 7.5782e-04
Validation auc = 0.698122
Epoch 170
Validation auc = 0.708451
Epoch 171
Validation auc = 0.716901
Epoch 172
Validation auc = 0.714554
Epoch 173
Validation auc = 0.715023
Epoch 174
Loss = 4.5371e-02, PNorm = 45.3019, GNorm = 0.5565, lr_0 = 7.5134e-04
Validation auc = 0.707981
Epoch 175
Validation auc = 0.700469
Epoch 176
Validation auc = 0.692019
Epoch 177
Validation auc = 0.687324
Epoch 178
Validation auc = 0.683568
Epoch 179
Loss = 1.3990e-01, PNorm = 45.3637, GNorm = 1.7278, lr_0 = 7.4492e-04
Validation auc = 0.691549
Epoch 180
Validation auc = 0.696244
Epoch 181
Validation auc = 0.691080
Epoch 182
Validation auc = 0.699061
Epoch 183
Validation auc = 0.710329
Epoch 184
Loss = 8.4965e-02, PNorm = 45.4301, GNorm = 1.9020, lr_0 = 7.3855e-04
Validation auc = 0.717840
Epoch 185
Validation auc = 0.713146
Epoch 186
Validation auc = 0.713146
Epoch 187
Validation auc = 0.713615
Epoch 188
Validation auc = 0.712676
Epoch 189
Loss = 1.1308e-01, PNorm = 45.4902, GNorm = 1.6598, lr_0 = 7.3224e-04
Validation auc = 0.717840
Epoch 190
Validation auc = 0.720188
Epoch 191
Validation auc = 0.719249
Epoch 192
Validation auc = 0.713146
Epoch 193
Validation auc = 0.714554
Epoch 194
Loss = 3.3201e-02, PNorm = 45.5475, GNorm = 0.6415, lr_0 = 7.2598e-04
Validation auc = 0.711268
Epoch 195
Validation auc = 0.706573
Epoch 196
Validation auc = 0.702817
Epoch 197
Validation auc = 0.700939
Epoch 198
Validation auc = 0.705634
Epoch 199
Loss = 8.5807e-02, PNorm = 45.6012, GNorm = 1.8528, lr_0 = 7.1977e-04
Validation auc = 0.715493
Epoch 200
Validation auc = 0.733803
Epoch 201
Validation auc = 0.741784
Epoch 202
Validation auc = 0.746009
Epoch 203
Validation auc = 0.749296
Epoch 204
Loss = 3.1230e-02, PNorm = 45.6456, GNorm = 1.3708, lr_0 = 7.1362e-04
Validation auc = 0.747887
Epoch 205
Validation auc = 0.744131
Epoch 206
Validation auc = 0.740376
Epoch 207
Validation auc = 0.731455
Epoch 208
Validation auc = 0.726761
Epoch 209
Loss = 2.0280e-02, PNorm = 45.7033, GNorm = 0.3592, lr_0 = 7.0752e-04
Validation auc = 0.726761
Epoch 210
Validation auc = 0.716901
Epoch 211
Validation auc = 0.717840
Epoch 212
Validation auc = 0.713615
Epoch 213
Validation auc = 0.716901
Epoch 214
Loss = 2.7328e-02, PNorm = 45.7582, GNorm = 0.6732, lr_0 = 7.0147e-04
Validation auc = 0.721127
Epoch 215
Validation auc = 0.720657
Epoch 216
Validation auc = 0.719718
Epoch 217
Validation auc = 0.716901
Epoch 218
Validation auc = 0.718779
Epoch 219
Loss = 4.2820e-02, PNorm = 45.8008, GNorm = 2.0717, lr_0 = 6.9548e-04
Validation auc = 0.719718
Epoch 220
Validation auc = 0.719249
Epoch 221
Validation auc = 0.714554
Epoch 222
Validation auc = 0.715493
Epoch 223
Validation auc = 0.725352
Epoch 224
Loss = 5.7787e-02, PNorm = 45.8471, GNorm = 1.8802, lr_0 = 6.8953e-04
Validation auc = 0.732864
Epoch 225
Validation auc = 0.724413
Epoch 226
Validation auc = 0.707981
Epoch 227
Validation auc = 0.705634
Epoch 228
Validation auc = 0.707981
Epoch 229
Loss = 5.0689e-02, PNorm = 45.9120, GNorm = 1.4194, lr_0 = 6.8364e-04
Validation auc = 0.716901
Epoch 230
Validation auc = 0.730047
Epoch 231
Validation auc = 0.731925
Epoch 232
Validation auc = 0.727700
Epoch 233
Validation auc = 0.725822
Epoch 234
Loss = 5.3868e-02, PNorm = 45.9662, GNorm = 2.1907, lr_0 = 6.7779e-04
Validation auc = 0.732864
Epoch 235
Validation auc = 0.734742
Epoch 236
Validation auc = 0.726291
Epoch 237
Validation auc = 0.715962
Epoch 238
Validation auc = 0.714085
Epoch 239
Loss = 4.7351e-02, PNorm = 46.0220, GNorm = 0.8262, lr_0 = 6.7200e-04
Validation auc = 0.715962
Epoch 240
Validation auc = 0.707981
Epoch 241
Validation auc = 0.695775
Epoch 242
Validation auc = 0.692488
Epoch 243
Validation auc = 0.696244
Epoch 244
Loss = 4.7540e-02, PNorm = 46.0795, GNorm = 0.3093, lr_0 = 6.6625e-04
Validation auc = 0.707512
Epoch 245
Validation auc = 0.715962
Epoch 246
Validation auc = 0.723005
Epoch 247
Validation auc = 0.723944
Epoch 248
Validation auc = 0.726761
Epoch 249
Loss = 7.4466e-02, PNorm = 46.1234, GNorm = 1.0483, lr_0 = 6.6056e-04
Validation auc = 0.730047
Epoch 250
Validation auc = 0.730986
Epoch 251
Validation auc = 0.733333
Epoch 252
Validation auc = 0.732864
Epoch 253
Validation auc = 0.733803
Epoch 254
Loss = 2.9152e-02, PNorm = 46.1734, GNorm = 0.6237, lr_0 = 6.5491e-04
Validation auc = 0.733803
Epoch 255
Validation auc = 0.728169
Epoch 256
Validation auc = 0.730986
Epoch 257
Validation auc = 0.734742
Epoch 258
Validation auc = 0.735211
Epoch 259
Loss = 4.7003e-02, PNorm = 46.2153, GNorm = 0.7236, lr_0 = 6.4931e-04
Validation auc = 0.734272
Epoch 260
Validation auc = 0.734742
Epoch 261
Validation auc = 0.732394
Epoch 262
Validation auc = 0.721596
Epoch 263
Validation auc = 0.715023
Epoch 264
Loss = 1.2559e-01, PNorm = 46.2486, GNorm = 1.6410, lr_0 = 6.4376e-04
Validation auc = 0.728638
Epoch 265
Validation auc = 0.735681
Epoch 266
Validation auc = 0.736150
Epoch 267
Validation auc = 0.736150
Epoch 268
Validation auc = 0.732394
Epoch 269
Loss = 9.1191e-02, PNorm = 46.2899, GNorm = 1.7467, lr_0 = 6.3826e-04
Validation auc = 0.737089
Epoch 270
Validation auc = 0.734742
Epoch 271
Validation auc = 0.742254
Epoch 272
Validation auc = 0.738498
Epoch 273
Validation auc = 0.736620
Epoch 274
Loss = 2.5447e-02, PNorm = 46.3358, GNorm = 1.3292, lr_0 = 6.3280e-04
Validation auc = 0.738498
Epoch 275
Validation auc = 0.739437
Epoch 276
Validation auc = 0.737559
Epoch 277
Validation auc = 0.736620
Epoch 278
Validation auc = 0.733803
Epoch 279
Loss = 3.9693e-02, PNorm = 46.3773, GNorm = 1.2530, lr_0 = 6.2739e-04
Validation auc = 0.731925
Epoch 280
Validation auc = 0.732864
Epoch 281
Validation auc = 0.730986
Epoch 282
Validation auc = 0.727700
Epoch 283
Validation auc = 0.729108
Epoch 284
Loss = 1.0494e-02, PNorm = 46.4124, GNorm = 0.2342, lr_0 = 6.2203e-04
Validation auc = 0.727230
Epoch 285
Validation auc = 0.730986
Epoch 286
Validation auc = 0.728638
Epoch 287
Validation auc = 0.722066
Epoch 288
Validation auc = 0.721127
Epoch 289
Loss = 8.9349e-03, PNorm = 46.4361, GNorm = 0.3744, lr_0 = 6.1671e-04
Validation auc = 0.717840
Epoch 290
Validation auc = 0.718779
Epoch 291
Validation auc = 0.718779
Epoch 292
Validation auc = 0.713615
Epoch 293
Validation auc = 0.711737
Epoch 294
Loss = 7.9706e-02, PNorm = 46.4578, GNorm = 2.0000, lr_0 = 6.1144e-04
Validation auc = 0.715962
Epoch 295
Validation auc = 0.721596
Epoch 296
Validation auc = 0.722535
Epoch 297
Validation auc = 0.725352
Epoch 298
Validation auc = 0.720188
Epoch 299
Loss = 3.1117e-03, PNorm = 46.4852, GNorm = 0.0616, lr_0 = 6.0621e-04
Validation auc = 0.721127
Model 0 best validation auc = 0.787793 on epoch 41
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.885611
Ensemble test auc = 0.885611
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=500, bias=False)
        (W_h): Linear(in_features=500, out_features=500, bias=False)
        (W_o): Linear(in_features=633, out_features=500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=500, out_features=500, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=500, out_features=1, bias=True)
  )
)
Number of parameters = 891,501
Moving model to cuda
Epoch 0
Validation auc = 0.616901
Epoch 1
Validation auc = 0.597183
Epoch 2
Validation auc = 0.627230
Epoch 3
Validation auc = 0.576056
Epoch 4
Loss = 6.7882e-01, PNorm = 42.2839, GNorm = 0.9178, lr_0 = 6.5000e-04
Validation auc = 0.553991
Epoch 5
Validation auc = 0.565728
Epoch 6
Validation auc = 0.576526
Epoch 7
Validation auc = 0.624883
Epoch 8
Validation auc = 0.640845
Epoch 9
Loss = 6.6683e-01, PNorm = 42.3478, GNorm = 1.1475, lr_0 = 9.9743e-04
Validation auc = 0.634742
Epoch 10
Validation auc = 0.602347
Epoch 11
Validation auc = 0.602817
Epoch 12
Validation auc = 0.618310
Epoch 13
Validation auc = 0.651643
Epoch 14
Loss = 6.1285e-01, PNorm = 42.4492, GNorm = 3.8620, lr_0 = 9.8890e-04
Validation auc = 0.694836
Epoch 15
Validation auc = 0.705634
Epoch 16
Validation auc = 0.687324
Epoch 17
Validation auc = 0.709390
Epoch 18
Validation auc = 0.692958
Epoch 19
Loss = 5.4040e-01, PNorm = 42.5313, GNorm = 0.9696, lr_0 = 9.8045e-04
Validation auc = 0.665728
Epoch 20
Validation auc = 0.706573
Epoch 21
Validation auc = 0.729577
Epoch 22
Validation auc = 0.702817
Epoch 23
Validation auc = 0.712676
Epoch 24
Loss = 5.2106e-01, PNorm = 42.6123, GNorm = 0.8232, lr_0 = 9.7207e-04
Validation auc = 0.733803
Epoch 25
Validation auc = 0.724413
Epoch 26
Validation auc = 0.732864
Epoch 27
Validation auc = 0.763380
Epoch 28
Validation auc = 0.730986
Epoch 29
Loss = 5.4960e-01, PNorm = 42.6915, GNorm = 4.0293, lr_0 = 9.6376e-04
Validation auc = 0.736150
Epoch 30
Validation auc = 0.780751
Epoch 31
Validation auc = 0.750704
Epoch 32
Validation auc = 0.724883
Epoch 33
Validation auc = 0.757746
Epoch 34
Loss = 4.8522e-01, PNorm = 42.7634, GNorm = 0.6921, lr_0 = 9.5552e-04
Validation auc = 0.771362
Epoch 35
Validation auc = 0.771362
Epoch 36
Validation auc = 0.759155
Epoch 37
Validation auc = 0.754930
Epoch 38
Validation auc = 0.775587
Epoch 39
Loss = 5.2475e-01, PNorm = 42.8505, GNorm = 3.0343, lr_0 = 9.4735e-04
Validation auc = 0.778873
Epoch 40
Validation auc = 0.770423
Epoch 41
Validation auc = 0.768545
Epoch 42
Validation auc = 0.776056
Epoch 43
Validation auc = 0.783568
Epoch 44
Loss = 5.0341e-01, PNorm = 42.9345, GNorm = 0.6700, lr_0 = 9.3925e-04
Validation auc = 0.781221
Epoch 45
Validation auc = 0.774648
Epoch 46
Validation auc = 0.780751
Epoch 47
Validation auc = 0.786385
Epoch 48
Validation auc = 0.784038
Epoch 49
Loss = 3.7643e-01, PNorm = 43.0353, GNorm = 0.9476, lr_0 = 9.3122e-04
Validation auc = 0.782629
Epoch 50
Validation auc = 0.783099
Epoch 51
Validation auc = 0.779343
Epoch 52
Validation auc = 0.772770
Epoch 53
Validation auc = 0.781221
Epoch 54
Loss = 3.4869e-01, PNorm = 43.1326, GNorm = 0.8219, lr_0 = 9.2326e-04
Validation auc = 0.785915
Epoch 55
Validation auc = 0.784977
Epoch 56
Validation auc = 0.782160
Epoch 57
Validation auc = 0.784038
Epoch 58
Validation auc = 0.778404
Epoch 59
Loss = 3.4560e-01, PNorm = 43.2303, GNorm = 1.3131, lr_0 = 9.1537e-04
Validation auc = 0.768075
Epoch 60
Validation auc = 0.769484
Epoch 61
Validation auc = 0.762441
Epoch 62
Validation auc = 0.752582
Epoch 63
Validation auc = 0.760563
Epoch 64
Loss = 3.8062e-01, PNorm = 43.3347, GNorm = 0.7635, lr_0 = 9.0754e-04
Validation auc = 0.754460
Epoch 65
Validation auc = 0.766667
Epoch 66
Validation auc = 0.759155
Epoch 67
Validation auc = 0.758216
Epoch 68
Validation auc = 0.745070
Epoch 69
Loss = 2.9806e-01, PNorm = 43.4483, GNorm = 2.5413, lr_0 = 8.9978e-04
Validation auc = 0.729108
Epoch 70
Validation auc = 0.737559
Epoch 71
Validation auc = 0.737559
Epoch 72
Validation auc = 0.742254
Epoch 73
Validation auc = 0.734742
Epoch 74
Loss = 3.0378e-01, PNorm = 43.5757, GNorm = 6.3726, lr_0 = 8.9209e-04
Validation auc = 0.720188
Epoch 75
Validation auc = 0.730516
Epoch 76
Validation auc = 0.715493
Epoch 77
Validation auc = 0.703756
Epoch 78
Validation auc = 0.710329
Epoch 79
Loss = 3.4555e-01, PNorm = 43.6797, GNorm = 1.1309, lr_0 = 8.8447e-04
Validation auc = 0.718779
Epoch 80
Validation auc = 0.729108
Epoch 81
Validation auc = 0.733803
Epoch 82
Validation auc = 0.751174
Epoch 83
Validation auc = 0.752113
Epoch 84
Loss = 2.4273e-01, PNorm = 43.7968, GNorm = 0.7089, lr_0 = 8.7691e-04
Validation auc = 0.742723
Epoch 85
Validation auc = 0.742254
Epoch 86
Validation auc = 0.740376
Epoch 87
Validation auc = 0.738498
Epoch 88
Validation auc = 0.715023
Epoch 89
Loss = 2.6412e-01, PNorm = 43.9071, GNorm = 1.7729, lr_0 = 8.6941e-04
Validation auc = 0.720657
Epoch 90
Validation auc = 0.713615
Epoch 91
Validation auc = 0.701408
Epoch 92
Validation auc = 0.701878
Epoch 93
Validation auc = 0.703286
Epoch 94
Loss = 2.2243e-01, PNorm = 44.0175, GNorm = 0.9107, lr_0 = 8.6198e-04
Validation auc = 0.679812
Epoch 95
Validation auc = 0.700000
Epoch 96
Validation auc = 0.688732
Epoch 97
Validation auc = 0.690610
Epoch 98
Validation auc = 0.699531
Epoch 99
Loss = 2.3508e-01, PNorm = 44.1201, GNorm = 3.6040, lr_0 = 8.5461e-04
Validation auc = 0.704225
Epoch 100
Validation auc = 0.697653
Epoch 101
Validation auc = 0.707512
Epoch 102
Validation auc = 0.721596
Epoch 103
Validation auc = 0.701408
Epoch 104
Loss = 2.1500e-01, PNorm = 44.2227, GNorm = 2.8596, lr_0 = 8.4730e-04
Validation auc = 0.695305
Epoch 105
Validation auc = 0.707512
Epoch 106
Validation auc = 0.700000
Epoch 107
Validation auc = 0.702347
Epoch 108
Validation auc = 0.714554
Epoch 109
Loss = 1.9353e-01, PNorm = 44.3212, GNorm = 3.6944, lr_0 = 8.4006e-04
Validation auc = 0.723944
Epoch 110
Validation auc = 0.724413
Epoch 111
Validation auc = 0.714085
Epoch 112
Validation auc = 0.709390
Epoch 113
Validation auc = 0.708920
Epoch 114
Loss = 1.8668e-01, PNorm = 44.4271, GNorm = 1.5881, lr_0 = 8.3288e-04
Validation auc = 0.707042
Epoch 115
Validation auc = 0.704695
Epoch 116
Validation auc = 0.702817
Epoch 117
Validation auc = 0.702817
Epoch 118
Validation auc = 0.704695
Epoch 119
Loss = 6.2291e-02, PNorm = 44.5299, GNorm = 0.9186, lr_0 = 8.2576e-04
Validation auc = 0.705164
Epoch 120
Validation auc = 0.707981
Epoch 121
Validation auc = 0.703756
Epoch 122
Validation auc = 0.692958
Epoch 123
Validation auc = 0.684507
Epoch 124
Loss = 1.1911e-01, PNorm = 44.6133, GNorm = 1.4436, lr_0 = 8.1870e-04
Validation auc = 0.678404
Epoch 125
Validation auc = 0.679343
Epoch 126
Validation auc = 0.681690
Epoch 127
Validation auc = 0.687324
Epoch 128
Validation auc = 0.703286
Epoch 129
Loss = 8.1207e-02, PNorm = 44.6829, GNorm = 1.1791, lr_0 = 8.1170e-04
Validation auc = 0.695775
Epoch 130
Validation auc = 0.702817
Epoch 131
Validation auc = 0.710329
Epoch 132
Validation auc = 0.698592
Epoch 133
Validation auc = 0.696714
Epoch 134
Loss = 9.8026e-02, PNorm = 44.7557, GNorm = 4.3405, lr_0 = 8.0476e-04
Validation auc = 0.714085
Epoch 135
Validation auc = 0.720657
Epoch 136
Validation auc = 0.703756
Epoch 137
Validation auc = 0.695305
Epoch 138
Validation auc = 0.694836
Epoch 139
Loss = 1.5103e-01, PNorm = 44.8419, GNorm = 5.4609, lr_0 = 7.9788e-04
Validation auc = 0.684507
Epoch 140
Validation auc = 0.677934
Epoch 141
Validation auc = 0.674648
Epoch 142
Validation auc = 0.685446
Epoch 143
Validation auc = 0.690610
Epoch 144
Loss = 7.1228e-02, PNorm = 44.9331, GNorm = 0.5550, lr_0 = 7.9106e-04
Validation auc = 0.695305
Epoch 145
Validation auc = 0.699531
Epoch 146
Validation auc = 0.705634
Epoch 147
Validation auc = 0.693897
Epoch 148
Validation auc = 0.687793
Epoch 149
Loss = 5.4791e-02, PNorm = 45.0075, GNorm = 1.0962, lr_0 = 7.8430e-04
Validation auc = 0.692019
Epoch 150
Validation auc = 0.695775
Epoch 151
Validation auc = 0.698122
Epoch 152
Validation auc = 0.695305
Epoch 153
Validation auc = 0.691549
Epoch 154
Loss = 9.6775e-02, PNorm = 45.0659, GNorm = 1.5142, lr_0 = 7.7759e-04
Validation auc = 0.681690
Epoch 155
Validation auc = 0.669953
Epoch 156
Validation auc = 0.669484
Epoch 157
Validation auc = 0.668545
Epoch 158
Validation auc = 0.679343
Epoch 159
Loss = 1.1227e-01, PNorm = 45.1221, GNorm = 1.3328, lr_0 = 7.7095e-04
Validation auc = 0.687793
Epoch 160
Validation auc = 0.691080
Epoch 161
Validation auc = 0.690141
Epoch 162
Validation auc = 0.688732
Epoch 163
Validation auc = 0.688263
Epoch 164
Loss = 4.9056e-02, PNorm = 45.1803, GNorm = 0.5878, lr_0 = 7.6436e-04
Validation auc = 0.682160
Epoch 165
Validation auc = 0.682629
Epoch 166
Validation auc = 0.709390
Epoch 167
Validation auc = 0.707981
Epoch 168
Validation auc = 0.696714
Epoch 169
Loss = 6.5702e-02, PNorm = 45.2449, GNorm = 1.5401, lr_0 = 7.5782e-04
Validation auc = 0.692488
Epoch 170
Validation auc = 0.695305
Epoch 171
Validation auc = 0.696244
Epoch 172
Validation auc = 0.700469
Epoch 173
Validation auc = 0.711737
Epoch 174
Loss = 9.4817e-02, PNorm = 45.3048, GNorm = 1.0122, lr_0 = 7.5134e-04
Validation auc = 0.713615
Epoch 175
Validation auc = 0.706573
Epoch 176
Validation auc = 0.693897
Epoch 177
Validation auc = 0.690610
Epoch 178
Validation auc = 0.688263
Epoch 179
Loss = 9.8374e-02, PNorm = 45.3709, GNorm = 0.8569, lr_0 = 7.4492e-04
Validation auc = 0.677934
Epoch 180
Validation auc = 0.674178
Epoch 181
Validation auc = 0.676526
Epoch 182
Validation auc = 0.679343
Epoch 183
Validation auc = 0.684038
Epoch 184
Loss = 1.0118e-01, PNorm = 45.4321, GNorm = 1.3855, lr_0 = 7.3855e-04
Validation auc = 0.685915
Epoch 185
Validation auc = 0.688263
Epoch 186
Validation auc = 0.692019
Epoch 187
Validation auc = 0.693427
Epoch 188
Validation auc = 0.689671
Epoch 189
Loss = 5.8313e-02, PNorm = 45.4960, GNorm = 0.8687, lr_0 = 7.3224e-04
Validation auc = 0.682160
Epoch 190
Validation auc = 0.689202
Epoch 191
Validation auc = 0.709390
Epoch 192
Validation auc = 0.708920
Epoch 193
Validation auc = 0.707512
Epoch 194
Loss = 3.8559e-02, PNorm = 45.5666, GNorm = 0.8473, lr_0 = 7.2598e-04
Validation auc = 0.713615
Epoch 195
Validation auc = 0.715962
Epoch 196
Validation auc = 0.717840
Epoch 197
Validation auc = 0.716901
Epoch 198
Validation auc = 0.709859
Epoch 199
Loss = 5.7684e-02, PNorm = 45.6362, GNorm = 1.4568, lr_0 = 7.1977e-04
Validation auc = 0.707512
Epoch 200
Validation auc = 0.704695
Epoch 201
Validation auc = 0.706103
Epoch 202
Validation auc = 0.707512
Epoch 203
Validation auc = 0.703286
Epoch 204
Loss = 1.3791e-01, PNorm = 45.6968, GNorm = 1.0170, lr_0 = 7.1362e-04
Validation auc = 0.706573
Epoch 205
Validation auc = 0.706103
Epoch 206
Validation auc = 0.707512
Epoch 207
Validation auc = 0.713615
Epoch 208
Validation auc = 0.725352
Epoch 209
Loss = 2.4387e-02, PNorm = 45.7473, GNorm = 0.6769, lr_0 = 7.0752e-04
Validation auc = 0.723474
Epoch 210
Validation auc = 0.724413
Epoch 211
Validation auc = 0.722066
Epoch 212
Validation auc = 0.726291
Epoch 213
Validation auc = 0.714085
Epoch 214
Loss = 5.0232e-02, PNorm = 45.7927, GNorm = 0.8115, lr_0 = 7.0147e-04
Validation auc = 0.698592
Epoch 215
Validation auc = 0.693427
Epoch 216
Validation auc = 0.692958
Epoch 217
Validation auc = 0.684507
Epoch 218
Validation auc = 0.687324
Epoch 219
Loss = 1.8197e-02, PNorm = 45.8408, GNorm = 0.2682, lr_0 = 6.9548e-04
Validation auc = 0.687324
Epoch 220
Validation auc = 0.688263
Epoch 221
Validation auc = 0.691549
Epoch 222
Validation auc = 0.689202
Epoch 223
Validation auc = 0.689671
Epoch 224
Loss = 6.8066e-02, PNorm = 45.8839, GNorm = 1.5914, lr_0 = 6.8953e-04
Validation auc = 0.695775
Epoch 225
Validation auc = 0.700939
Epoch 226
Validation auc = 0.689671
Epoch 227
Validation auc = 0.686385
Epoch 228
Validation auc = 0.693897
Epoch 229
Loss = 1.6725e-02, PNorm = 45.9285, GNorm = 0.9268, lr_0 = 6.8364e-04
Validation auc = 0.704225
Epoch 230
Validation auc = 0.709390
Epoch 231
Validation auc = 0.712207
Epoch 232
Validation auc = 0.714554
Epoch 233
Validation auc = 0.711268
Epoch 234
Loss = 1.5864e-02, PNorm = 45.9691, GNorm = 0.7909, lr_0 = 6.7779e-04
Validation auc = 0.713146
Epoch 235
Validation auc = 0.713146
Epoch 236
Validation auc = 0.711502
Epoch 237
Validation auc = 0.711737
Epoch 238
Validation auc = 0.720188
Epoch 239
Loss = 3.2471e-02, PNorm = 46.0034, GNorm = 1.1873, lr_0 = 6.7200e-04
Validation auc = 0.723005
Epoch 240
Validation auc = 0.704225
Epoch 241
Validation auc = 0.690141
Epoch 242
Validation auc = 0.672300
Epoch 243
Validation auc = 0.661033
Epoch 244
Loss = 4.4205e-02, PNorm = 46.0498, GNorm = 1.8872, lr_0 = 6.6625e-04
Validation auc = 0.659624
Epoch 245
Validation auc = 0.664789
Epoch 246
Validation auc = 0.690141
Epoch 247
Validation auc = 0.715962
Epoch 248
Validation auc = 0.708451
Epoch 249
Loss = 6.7884e-02, PNorm = 46.0865, GNorm = 2.1424, lr_0 = 6.6056e-04
Validation auc = 0.705164
Epoch 250
Validation auc = 0.686385
Epoch 251
Validation auc = 0.667606
Epoch 252
Validation auc = 0.669484
Epoch 253
Validation auc = 0.670892
Epoch 254
Loss = 5.5983e-02, PNorm = 46.1387, GNorm = 1.4108, lr_0 = 6.5491e-04
Validation auc = 0.678404
Epoch 255
Validation auc = 0.687793
Epoch 256
Validation auc = 0.696714
Epoch 257
Validation auc = 0.700469
Epoch 258
Validation auc = 0.706103
Epoch 259
Loss = 8.5607e-02, PNorm = 46.1834, GNorm = 0.8818, lr_0 = 6.4931e-04
Validation auc = 0.703756
Epoch 260
Validation auc = 0.702817
Epoch 261
Validation auc = 0.702817
Epoch 262
Validation auc = 0.700939
Epoch 263
Validation auc = 0.706103
Epoch 264
Loss = 2.8938e-02, PNorm = 46.2325, GNorm = 0.2605, lr_0 = 6.4376e-04
Validation auc = 0.707042
Epoch 265
Validation auc = 0.711268
Epoch 266
Validation auc = 0.719249
Epoch 267
Validation auc = 0.721127
Epoch 268
Validation auc = 0.721127
Epoch 269
Loss = 3.6024e-02, PNorm = 46.2777, GNorm = 0.4647, lr_0 = 6.3826e-04
Validation auc = 0.711268
Epoch 270
Validation auc = 0.706103
Epoch 271
Validation auc = 0.700469
Epoch 272
Validation auc = 0.690610
Epoch 273
Validation auc = 0.680282
Epoch 274
Loss = 7.0831e-02, PNorm = 46.3211, GNorm = 1.3619, lr_0 = 6.3280e-04
Validation auc = 0.678404
Epoch 275
Validation auc = 0.685915
Epoch 276
Validation auc = 0.694366
Epoch 277
Validation auc = 0.701878
Epoch 278
Validation auc = 0.703756
Epoch 279
Loss = 2.3528e-02, PNorm = 46.3608, GNorm = 0.3057, lr_0 = 6.2739e-04
Validation auc = 0.707042
Epoch 280
Validation auc = 0.704695
Epoch 281
Validation auc = 0.703286
Epoch 282
Validation auc = 0.703756
Epoch 283
Validation auc = 0.704695
Epoch 284
Loss = 5.0245e-02, PNorm = 46.3926, GNorm = 1.3607, lr_0 = 6.2203e-04
Validation auc = 0.693897
Epoch 285
Validation auc = 0.692488
Epoch 286
Validation auc = 0.695775
Epoch 287
Validation auc = 0.702347
Epoch 288
Validation auc = 0.706573
Epoch 289
Loss = 1.5701e-02, PNorm = 46.4282, GNorm = 0.6822, lr_0 = 6.1671e-04
Validation auc = 0.705164
Epoch 290
Validation auc = 0.708920
Epoch 291
Validation auc = 0.708920
Epoch 292
Validation auc = 0.716432
Epoch 293
Validation auc = 0.719718
Epoch 294
Loss = 7.8657e-02, PNorm = 46.4569, GNorm = 3.5399, lr_0 = 6.1144e-04
Validation auc = 0.720188
Epoch 295
Validation auc = 0.713615
Epoch 296
Validation auc = 0.712207
Epoch 297
Validation auc = 0.716432
Epoch 298
Validation auc = 0.718779
Epoch 299
Loss = 1.8286e-02, PNorm = 46.4937, GNorm = 0.8756, lr_0 = 6.0621e-04
Validation auc = 0.721596
Model 0 best validation auc = 0.786385 on epoch 47
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.909693
Ensemble test auc = 0.909693
10-fold cross validation
	Seed 0 ==> test auc = 0.889223
	Seed 1 ==> test auc = 0.892234
	Seed 2 ==> test auc = 0.894040
	Seed 3 ==> test auc = 0.886815
	Seed 4 ==> test auc = 0.889223
	Seed 5 ==> test auc = 0.893438
	Seed 6 ==> test auc = 0.886815
	Seed 7 ==> test auc = 0.863937
	Seed 8 ==> test auc = 0.885611
	Seed 9 ==> test auc = 0.909693
Overall test auc = 0.889103 +/- 0.010644
Elapsed time = 0:05:05
